- 02 Dec, 2022 25 commits
-
-
John Ogness authored
It is not reliable to check for CON_ENABLED in order to identify if a console is registered. Use console_is_registered() instead. Signed-off-by:
John Ogness <john.ogness@linutronix.de> Reviewed-by:
Petr Mladek <pmladek@suse.com> Signed-off-by:
-
John Ogness authored
The CON_ENABLED status of a console is a runtime setting that does not involve the console driver. Drivers must not assume that if the console is disabled then proper hardware management is not needed. For the EFI earlycon case, it is about remapping/unmapping memory for the framebuffer. Use console_is_registered() instead of checking CON_ENABLED. Signed-off-by:
-
John Ogness authored
Currently CON_ENABLED is being (mis)used to identify if the console has been registered. This is not reliable because it can be set even though registration failed or it can be unset, even though the console is registered. Use console_is_registered() instead. Signed-off-by:
Geert Uytterhoeven <geert@linux-m68k.org> Acked-by:
-
John Ogness authored
All users of uart_console_enabled() really want to know if a console is registered. It is not reliable to check for CON_ENABLED in order to identify if a console is registered. Use console_is_registered() instead. A _locked() variant is provided because uart_set_options() is always called with the console_list_lock held and must check if a console is registered in order to synchronize with kgdboc. Signed-off-by:
-
John Ogness authored
Currently it is not possible for drivers to detect if they have already successfully registered their console. Several drivers have multiple paths that lead to console registration. To avoid attempting a 2nd registration (which leads to a WARN), drivers are implementing their own solution. Introduce console_is_registered() so drivers can easily identify if their console is currently registered. A _locked() variant is also provided if the caller is already holding the console_list_lock. Signed-off-by:
-
John Ogness authored
Use srcu console list iteration for console list traversal. It is acceptable because the consoles might come and go at any time. Strict synchronizing with console registration code would not bring any advantage over srcu. Document why the console_lock is still necessary. Note that this is a preparatory change for when console_lock no longer provides synchronization for the console list. Signed-off-by:
-
John Ogness authored
With SRCU it is now safe to traverse the console list, even if the console_trylock() failed. However, overwriting console->seq when console_trylock() failed is still an issue. Switch to SRCU iteration and document remaining issue with console->seq. Signed-off-by:
-
John Ogness authored
Use srcu console list iteration for console list traversal. Document why the console_lock is still necessary. Note that this is a preparatory change for when console_lock no longer provides synchronization for the console list. Signed-off-by:
-
John Ogness authored
All users of console_is_usable() are SRCU iterators. Use the appropriate wrapper function to locklessly read the flags. Signed-off-by:
-
John Ogness authored
Use srcu console list iteration for console list traversal. Document why the console_lock is still necessary. Note that this is a preparatory change for when console_lock no longer provides synchronization for the console list. Signed-off-by:
-
John Ogness authored
Guarantee safe iteration of the console list by using SRCU. Note that in the case of a handover, the SRCU read lock is also released. This is documented in the function description and as comments in the code. It is a bit tricky, but this preserves the lockdep lock ordering for the context handing over the console_lock: console_lock() | mutex_acquire(&console_lock_dep_map) <-- console lock | console_unlock() | console_flush_all() | | srcu_read_lock(&console_srcu) <-- srcu lock | | console_emit_next_record() | | | console_lock_spinning_disable_and_check() | | | | srcu_read_unlock(&console_srcu) <-- srcu unlock | | | | mutex_release(&console_lock_dep_map) <-- console unlock Signed-off-by:
-
John Ogness authored
Guarantee safe iteration of the console list by using SRCU. Signed-off-by:
Aaron Tomlin <atomlin@atomlin.com> Reviewed-by:
Douglas Anderson <dianders@chromium.org> Signed-off-by:
-
John Ogness authored
Rather than using the console_lock to guarantee safe console list traversal, use srcu console list iteration. Signed-off-by:
-
John Ogness authored
After switching to SRCU for console list iteration, some readers will begin readings console->flags as a data race. Locklessly reading console->flags provides a consistent value because there is at most one CPU modifying console->flags and that CPU is using only read-modify-write operations. Introduce a wrapper for SRCU iterators to read console flags. Introduce a matching wrapper to write to flags of registered consoles. Writing to flags of registered consoles is synchronized by the console_list_lock. Signed-off-by:
-
John Ogness authored
Currently there exist races in register_console(), where the types of registered consoles are checked (without holding the console_lock) and then after acquiring the console_lock, it is assumed that the list has not changed. Also, some code that performs console_unregister() make similar assumptions. It might be possible to fix these races using the console_lock. But it would require a complex analysis of all console drivers to make sure that the console_lock is not taken in match() and setup() callbacks. And we really prefer to split up and reduce the responsibilities of console_lock rather than expand its complexity. Therefore, introduce a new console_list_lock to provide full synchronization for any console list changes. In addition, also use console_list_lock for synchronization of console->flags updates. All flags are either static or modified only during the console registration. There are only two exceptions. The first exception is CON_ENABLED, which is also modified by console_start()/console_stop(). Therefore, these functions must also take the console_list_lock. The second exception is when the flags are modified by the console driver init code before the console is registered. These will be ignored because they are not visible to the rest of the system via the console_drivers list. Note that one of the various responsibilities of the console_lock is also intended to provide console list and console->flags synchronization. Later changes will update call sites relying on the console_lock for these purposes. Once all call sites have been updated, the console_lock will be relieved of synchronizing console_list and console->flags updates. Suggested-by:
Thomas Gleixner <tglx@linutronix.de> Signed-off-by:
-
John Ogness authored
The console_lock is held throughout the start/show/stop procedure to print out device/driver information about all registered consoles. Since the console_lock is being used for multiple reasons, explicitly document these reasons. This will be useful when the console_lock is split into fine-grained locking. Signed-off-by:
-
John Ogness authored
show_cons_active() uses the console_lock to gather information on registered consoles. Since the console_lock is being used for multiple reasons, explicitly document these reasons. This will be useful when the console_lock is split into fine-grained locking. Signed-off-by:
-
John Ogness authored
kgdboc_earlycon_init() uses the console_lock to ensure that no consoles are unregistered until the kgdboc_earlycon is setup. This is necessary because the trapping of the exit() callback assumes that the exit() callback is not called before the trap is setup. Explicitly document this non-typical console_lock usage. Signed-off-by:
Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by:
Daniel Thompson <daniel.thompson@linaro.org> Reviewed-by:
-
John Ogness authored
The initial intention of the UML kmsg_dumper is to dump the kernel buffers to stdout if there is no console available to perform the regular crash output. However, if ttynull was registered as a console, no crash output was seen. Commit e23fe90d ("um: kmsg_dumper: always dump when not tty console") tried to fix this by performing the kmsg_dump unless the stdio console was behind /dev/console or enabled. But this allowed kmsg dumping to occur even if other non-stdio consoles will output the crash output. Also, a console being the driver behind /dev/console has nothing to do with a crash scenario. Restore the initial intention by dumping the kernel buffers to stdout only if a non-ttynull console is registered and enabled. Also add detailed comments so that it is clear why these rules are applied. Signed-off-by:
-
John Ogness authored
It used to be that all consoles were synchronized with respect to which message they were printing. After commit a699449b ("printk: refactor and rework printing logic"), all consoles have their own @seq for tracking which message they are on. That commit also changed how the initial sequence number was chosen. Instead of choosing the next non-printed message, it chose the sequence number of the next message that will be added to the ringbuffer. That change created a possibility that a non-boot console taking over for a boot console might skip messages if the boot console was behind and did not have a chance to catch up before being unregistered. Since it is not known which boot console is the same device, flush all consoles and, if necessary, start with the message of the enabled boot console that is the furthest behind. If no boot consoles are enabled, begin with the next message that will be added to the ringbuffer. Also, since boot consoles are meant to be used at boot time, handle them the same as CON_PRINTBUFFER to ensure that no initial messages are skipped. Signed-off-by:
-
John Ogness authored
The code to initialize @seq for a new console needs to consider more factors when choosing an initial value. Move the code into a helper function console_init_seq() "as is" so this code can be expanded without causing register_console() to become too long. A later commit will implement the additional code. Signed-off-by:
-
John Ogness authored
The @bootcon_enabled and @realcon_enabled local variables actually represent if such console types are registered. In general there has been a confusion about enabled vs. registered. Incorrectly naming such variables promotes such confusion. Rename the variables to _registered. Signed-off-by:
-
John Ogness authored
Provide an NMI-safe SRCU protected variant to walk the console list. Note that all console fields are now set before adding the console to the list to avoid the console becoming visible by SCRU readers before being fully initialized. This is a preparatory change for a new console infrastructure which operates independent of the console BKL. Suggested-by:
Miguel Ojeda <ojeda@kernel.org> Reviewed-by:
Paul E. McKenney <paulmck@kernel.org> Reviewed-by:
-
Thomas Gleixner authored
Replace the open coded single linked list with a hlist so a conversion to SRCU protected list walks can reuse the existing primitives. Co-developed-by:
Sergey Senozhatsky <senozhatsky@chromium.org> Reviewed-by:
-
Thomas Gleixner authored
Unprotected list walks are not necessarily safe. Signed-off-by:
-
- 01 Dec, 2022 1 commit
-
-
Zqiang authored
This commit adds lockdep checks for illegal use of synchronize_srcu() within same-type SRCU read-side critical sections and within normal RCU read-side critical sections. It also makes synchronize_srcu() be a no-op during early boot. These changes bring Tiny synchronize_srcu() into line with both Tree synchronize_srcu() and Tiny synchronize_rcu(). Signed-off-by:
Zqiang <qiang1.zhang@intel.com> Signed-off-by:
-
- 29 Nov, 2022 1 commit
-
-
Paul E. McKenney authored
Kernels configured with CONFIG_PRINTK=n and CONFIG_SRCU=n get build failures. This causes trouble for deep embedded systems. But given that there are more than 25 instances of "select SRCU" in the kernel, it is hard to believe that there are many kernels running in production without SRCU. This commit therefore makes SRCU mandatory. The SRCU Kconfig option remains for backwards compatibility, and will be removed when it is no longer used. [ paulmck: Update per kernel test robot feedback. ] Reported-by:
-
- 09 Nov, 2022 1 commit
-
-
John Ogness authored
Provide an implementation for debug_lockdep_rcu_enabled() when CONFIG_DEBUG_LOCK_ALLOC is not enabled. This allows code to check if rcu lockdep debugging is available without needing an extra check if CONFIG_DEBUG_LOCK_ALLOC is enabled. Signed-off-by:
-
- 21 Oct, 2022 5 commits
-
-
Frederic Weisbecker authored
Currently the NMI safety debugging is only performed on architectures that don't support NMI-safe this_cpu_inc(). Reorder the code so that other architectures like x86 also detect bad uses. [ paulmck: Apply kernel test robot, Stephen Rothwell, and Zqiang feedback. ] Signed-off-by:
Frederic Weisbecker <frederic@kernel.org> Signed-off-by:
-
Frederic Weisbecker authored
Tell about the need to protect against concurrent updaters who may overflow the GP counter behind the current update. Reported-by:
-
Frederic Weisbecker authored
Using the NMI-unsafe reader API from within an NMI handler is very likely to be buggy for three reasons: 1) NMIs aren't strictly re-entrant (a pending nested NMI will execute at the end of the current one) so it should be fine to use a non-atomic increment here. However, breakpoints can still interrupt NMIs and if a breakpoint callback has a reader on that same ssp, a racy increment can happen. 2) If the only reader site for a given srcu_struct structure is in an NMI handler, then RCU should be used instead of SRCU. 3) Because of the previous reason (2), an srcu_struct structure having an SRCU read side critical section in an NMI handler is likely to have another one from a task context. For all these reasons, warn if an NMI-unsafe reader API is used from an NMI handler. Signed-off-by:
Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by:
-
Paul E. McKenney authored
The s390 architecture uses either a cmpxchg loop (old systems) or the laa add-to-memory instruction (new systems) to implement this_cpu_add(), both of which are NMI safe. This means that the old and more-efficient srcu_read_lock() may be used in NMI context, without the need for srcu_read_lock_nmisafe(). Therefore, add the new Kconfig option ARCH_HAS_NMI_SAFE_THIS_CPU_OPS to arch/s390/Kconfig, which will cause NEED_SRCU_NMI_SAFE to be deselected, thus preserving the current srcu_read_lock() behavior. [ paulmck: Apply Christian Borntraeger feedback. ] Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Suggested-by:
Neeraj Upadhyay <quic_neeraju@quicinc.com> Suggested-by:
Boqun Feng <boqun.feng@gmail.com> Signed-off-by:
Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Sven Schnelle <svens@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: John Ogness <john.ogness@linutronix.de> Cc: Petr Mladek <pmladek@suse.com> Cc: <linux-s390@vger.kernel.org>
-
Paul E. McKenney authored
The loongarch architecture uses the atomic read-modify-write amadd instruction to implement this_cpu_add(), which is NMI safe. This means that the old and more-efficient srcu_read_lock() may be used in NMI context, without the need for srcu_read_lock_nmisafe(). Therefore, add the new Kconfig option ARCH_HAS_NMI_SAFE_THIS_CPU_OPS to arch/loongarch/Kconfig, which will cause NEED_SRCU_NMI_SAFE to be deselected, thus preserving the current srcu_read_lock() behavior. Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Suggested-by:
-
- 20 Oct, 2022 5 commits
-
-
Paul E. McKenney authored
The arm64 architecture uses either an LL/SC loop (old systems) or an LSE stadd instruction (new systems) to implement this_cpu_add(), both of which are NMI safe. This means that the old and more-efficient srcu_read_lock() may be used in NMI context, without the need for srcu_read_lock_nmisafe(). Therefore, add the new Kconfig option ARCH_HAS_NMI_SAFE_THIS_CPU_OPS to arch/arm64/Kconfig, which will cause NEED_SRCU_NMI_SAFE to be deselected, thus preserving the current srcu_read_lock() behavior. Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Suggested-by:
-
Paul E. McKenney authored
The x86 architecture uses an add-to-memory instruction to implement this_cpu_add(), which is NMI safe. This means that the old and more-efficient srcu_read_lock() may be used in NMI context, without the need for srcu_read_lock_nmisafe(). Therefore, add the new Kconfig option ARCH_HAS_NMI_SAFE_THIS_CPU_OPS to arch/x86/Kconfig, which will cause NEED_SRCU_NMI_SAFE to be deselected, thus preserving the current srcu_read_lock() behavior. Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by:
-
Paul E. McKenney authored
This commit adds runtime checks to verify that a given srcu_struct uses consistent NMI-safe (or not) read-side primitives globally, but based on the per-CPU data. These global checks are made by the grace-period code that must scan the srcu_data structures anyway, and are done only in kernels built with CONFIG_PROVE_RCU=y. Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by:
-
Paul E. McKenney authored
This commit adds runtime checks to verify that a given srcu_struct uses consistent NMI-safe (or not) read-side primitives on a per-CPU basis. Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by:
-
Paul E. McKenney authored
On strict load-store architectures, the use of this_cpu_inc() by srcu_read_lock() and srcu_read_unlock() is not NMI-safe in TREE SRCU. To see this suppose that an NMI arrives in the middle of srcu_read_lock(), just after it has read ->srcu_lock_count, but before it has written the incremented value back to memory. If that NMI handler also does srcu_read_lock() and srcu_read_lock() on that same srcu_struct structure, then upon return from that NMI handler, the interrupted srcu_read_lock() will overwrite the NMI handler's update to ->srcu_lock_count, but leave unchanged the NMI handler's update by srcu_read_unlock() to ->srcu_unlock_count. This can result in a too-short SRCU grace period, which can in turn result in arbitrary memory corruption. If the NMI handler instead interrupts the srcu_read_unlock(), this can result in eternal SRCU grace periods, which is not much better. This commit therefore creates a pair of new srcu_read_lock_nmisafe() and srcu_read_unlock_nmisafe() functions, which allow SRCU readers in both NMI handlers and in process and IRQ context. It is bad practice to mix the existing and the new _nmisafe() primitives on the same srcu_struct structure. Use one set or the other, not both. Just to underline that "bad practice" point, using srcu_read_lock() at process level and srcu_read_lock_nmisafe() in your NMI handler will not, repeat NOT, work. If you do not immediately understand why this is the case, please review the earlier paragraphs in this commit log. [ paulmck: Apply kernel test robot feedback. ] [ paulmck: Apply feedback from Randy Dunlap. ] [ paulmck: Apply feedback from John Ogness. ] [ paulmck: Apply feedback from Frederic Weisbecker. ] Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by:
-
- 18 Oct, 2022 1 commit
-
-
Paul E. McKenney authored
NMI-safe variants of srcu_read_lock() and srcu_read_unlock() are needed by printk(), which on many architectures entails read-modify-write atomic operations. This commit prepares Tree SRCU for this change by making both ->srcu_lock_count and ->srcu_unlock_count by atomic_long_t. [ paulmck: Apply feedback from John Ogness. ] Link: https://lore.kernel.org/all/20220910221947.171557773@linutronix.de/Signed-off-by:
-
- 16 Oct, 2022 1 commit
-
-
Linus Torvalds authored
-
