Destroy current's io-wq backend and tctx on __io_uring_task_cancel(),
aka exec(). Looks it's not strictly necessary, because it will be done
at some point when the task dies and changes of creds/files/etc. are
handled, but better to do that earlier to free io-wq and not potentially
lock previous mm and other resources for the time being.

It's safe to do because we wait for all requests of the current task to
complete, so no request will use tctx afterwards. Note, that
io_uring_files_cancel() may leave some requests for later reaping, so it
leaves tctx intact, that's ok as the task is dying anyway.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Make sure that we killed an io-wq by the time a task is dead.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

We clear the bit marking the ctx task_work as active after having run
the queued work, but we really should be clearing it before. Otherwise
we can hit a tiny race ala:

CPU0					CPU1
io_task_work_add()			tctx_task_work()
					run_work
	add_to_list
	test_and_set_bit
					clear_bit
		already set

and CPU0 will return thinking the task_work is queued, while in reality
it's already being run. If we hit the condition after __tctx_task_work()
found no more work, but before we've cleared the bit, then we'll end up
thinking it's queued and will be run. In reality it is queued, but we
didn't queue the ctx task_work to ensure that it gets run.

Fixes: 7cbf1722 ("io_uring: provide FIFO ordering for task_work")
Signed-off-by: Jens Axboe <axboe@kernel.dk>

If we put the io-wq from io_uring, we really want it to exit. Provide
a helper that does that for us. Couple that with not having the manager
hold a reference to the 'wq' and the normal SQPOLL exit will tear down
the io-wq context appropriate.

On the io-wq side, our wq context is per task, so only the task itself
is manipulating ->manager and hence it's safe to check and clear without
any extra locking. We just need to ensure that the manager task stays
around, in case it exits.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

We want to reuse this completion, and a single complete should do just
fine. Ensure that we park ourselves first if requested, as that is what
lead to the initial deadlock in this area. If we've got someone attempting
to park us, then we can't proceed without having them finish first.

Fixes: 37d1e2e3 ("io_uring: move SQPOLL thread io-wq forked worker")
Signed-off-by: Jens Axboe <axboe@kernel.dk>

io_uring_try_cancel_requests() matches not only current's requests, but
also of other exiting tasks, so we need to actively cancel them and not
just wait, especially since the function can be called on flush during
do_exit() -> exit_files().
Even if it's not a problem for now, it's much nicer to know that the
function tries to cancel everything it can.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

If we fail to fork an SQPOLL worker, we can hit cancel, and hence
attempted thread stop, with the thread already being stopped. Ensure
we check for that.

Also guard thread stop fully by the sqd mutex, just like we do for
park.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

We are already freeing the wq struct in both spots, so don't put it and
get it freed twice.

Reported-by: syzbot+7bf785eedca35ca05501@syzkaller.appspotmail.com
Fixes: 4fb6ac32 ("io-wq: improve manager/worker handling over exec")
Signed-off-by: Jens Axboe <axboe@kernel.dk>

The manager waits for the workers, hence the manager is always valid if
workers are running. Now also have wq destroy wait for the manager on
exit, so we now everything is gone.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

This is a leftover from a different use cases, it's used to wait for
the manager to startup. Rename it as such.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

If we're in the process of shutting down the async context, then don't
create new workers if we already have at least the fixed one.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Instead of having to wait separately on workers and manager, just have
the manager wait on the workers. We use an atomic_t for the reference
here, as we need to start at 0 and allow increment from that. Since the
number of workers is naturally capped by the allowed nr of processes,
and that uses an int, there is no risk of overflow.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

We need to have our worker count updated before continuing, to avoid
cases where we repeatedly think we need a new worker, but a fork is
already in progress.
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Pull ide fix from Jens Axboe:
 "This is a leftover fix from 5.11, where I forgot to ship it your way"

* tag 'ide-5.11-2021-02-28' of git://git.kernel.dk/linux-block:
  ide/falconide: Fix module unload

Merge tag 'kbuild-fixes-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild fixes from Masahiro Yamada:

 - Fix UNUSED_KSYMS_WHITELIST for Clang LTO

 - Make -s builds really silent irrespective of V= option

 - Fix build error when SUBLEVEL or PATCHLEVEL is empty

* tag 'kbuild-fixes-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  kbuild: Fix <linux/version.h> for empty SUBLEVEL or PATCHLEVEL again
  kbuild: make -s option take precedence over V=1
  ia64: remove redundant READELF from arch/ia64/Makefile
  kbuild: do not include include/config/auto.conf from adjust_autoksyms.sh
  kbuild: fix UNUSED_KSYMS_WHITELIST for Clang LTO
  kbuild: lto: add _mcount to list of used symbols

Pull arch/csky updates from Guo Ren:
 "Features:
   - add new memory layout 2.5G(user):1.5G(kernel)
   - add kmemleak support
   - reconstruct VDSO framework: add VDSO with GENERIC_GETTIMEOFDAY,
     GENERIC_TIME_VSYSCALL, HAVE_GENERIC_VDSO
   - add faulthandler_disabled() check
   - support (fix) swapon
   - add (fix) _PAGE_ACCESSED for default pgprot
   - abort uaccess retries upon fatal signal (from arm)

  Fixes and optimizations:
   - fix perf probe failure
   - fix show_regs doesn't contain regs->usp
   - remove custom asm/atomic.h implementation
   - fix barrier design
   - fix futex SMP implementation
   - fix asm/cmpxchg.h with correct ordering barrier
   - cleanup asm/spinlock.h
   - fix PTE global for 2.5:1.5 virtual memory
   - remove prologue of page fault handler in entry.S
   - fix TLB maintenance synchronization problem
   - add show_tlb for CPU_CK860 debug
   - fix FAULT_FLAG_XXX param for handle_mm_fault
   - fix update_mmu_cache called with user io mapping
   - fix do_page_fault parent irq status
   - fix a size determination in gpr_get()
   - pgtable.h: Coding convention
   - kprobe: Fix code in simulate without 'long'
   - fix pfn_valid error with wrong max_mapnr
   - use free_initmem_default() in free_initmem()
   - fix compile error"

* tag 'csky-for-linus-5.12-rc1' of git://github.com/c-sky/csky-linux: (30 commits)
  csky: Fixup compile error
  csky: use free_initmem_default() in free_initmem()
  csky: Fixup pfn_valid error with wrong max_mapnr
  csky: Add VDSO with GENERIC_GETTIMEOFDAY, GENERIC_TIME_VSYSCALL, HAVE_GENERIC_VDSO
  csky: kprobe: Fixup code in simulate without 'long'
  csky: Fixup swapon
  csky: pgtable.h: Coding convention
  csky: Fixup _PAGE_ACCESSED for default pgprot
  csky: remove unused including <linux/version.h>
  csky: Fix a size determination in gpr_get()
  csky: Reconstruct VDSO framework
  csky: mm: abort uaccess retries upon fatal signal
  csky: Sync riscv mm/fault.c for easy maintenance
  csky: Fixup do_page_fault parent irq status
  csky: Add faulthandler_disabled() check
  csky: Fixup update_mmu_cache called with user io mapping
  csky: Fixup FAULT_FLAG_XXX param for handle_mm_fault
  csky: Add show_tlb for CPU_CK860 debug
  csky: Fix TLB maintenance synchronization problem
  csky: Add kmemleak support
  ...

Pull more RISC-V updates from Palmer Dabbelt:
 "A pair of patches that slipped through the cracks:

   - enable CPU hotplug in the defconfigs

   - some cleanups to setup_bootmem

  There's also a single fix for some randconfig build failures:

   - make NUMA depend on SMP"

* tag 'riscv-for-linus-5.12-mw1' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
  riscv: Cleanup setup_bootmem()
  RISC-V: Enable CPU Hotplug in defconfigs
  RISC-V: Make NUMA depend on SMP

Pull more SCSI updates from James Bottomley:
 "This is a few driver updates (iscsi, mpt3sas) that were still in the
  staging queue when the merge window opened (all committed on or before
  8 Feb) and some small bug fixes which came in during the merge window
  (all committed on 22 Feb)"

* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (30 commits)
  scsi: hpsa: Correct dev cmds outstanding for retried cmds
  scsi: sd: Fix Opal support
  scsi: target: tcmu: Fix memory leak caused by wrong uio usage
  scsi: target: tcmu: Move some functions without code change
  scsi: sd: sd_zbc: Don't pass GFP_NOIO to kvcalloc
  scsi: aic7xxx: Remove unused function pointer typedef ahc_bus_suspend/resume_t
  scsi: bnx2fc: Fix Kconfig warning & CNIC build errors
  scsi: ufs: Fix a duplicate dev quirk number
  scsi: aic79xx: Fix spelling of version
  scsi: target: core: Prevent underflow for service actions
  scsi: target: core: Add cmd length set before cmd complete
  scsi: iscsi: Drop session lock in iscsi_session_chkready()
  scsi: qla4xxx: Use iscsi_is_session_online()
  scsi: libiscsi: Reset max/exp cmdsn during recovery
  scsi: iscsi_tcp: Fix shost can_queue initialization
  scsi: libiscsi: Add helper to calculate max SCSI cmds per session
  scsi: libiscsi: Fix iSCSI host workq destruction
  scsi: libiscsi: Fix iscsi_task use after free()
  scsi: libiscsi: Drop taskqueuelock
  scsi: libiscsi: Fix iscsi_prep_scsi_cmd_pdu() error handling
  ...

Pull more xfs updates from Darrick Wong:
 "The most notable fix here prevents premature reuse of freed metadata
  blocks, and adding the ability to detect accidental nested
  transactions, which are not allowed here.

   - Restore a disused sysctl control knob that was inadvertently
     dropped during the merge window to avoid fstests regressions.

   - Don't speculatively release freed blocks from the busy list until
     we're actually allocating them, which fixes a rare log recovery
     regression.

   - Don't nest transactions when scanning for free space.

   - Add an idiot^Wmaintainer light to detect nested transactions. ;)"

* tag 'xfs-5.12-merge-6' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux:
  xfs: use current->journal_info for detecting transaction recursion
  xfs: don't nest transactions when scanning for eofblocks
  xfs: don't reuse busy extents on extent trim
  xfs: restore speculative_cow_prealloc_lifetime sysctl

Pull more block updates from Jens Axboe:
 "A few stragglers (and one due to me missing it originally), and fixes
  for changes in this merge window mostly. In particular:

   - blktrace cleanups (Chaitanya, Greg)

   - Kill dead blk_pm_* functions (Bart)

   - Fixes for the bio alloc changes (Christoph)

   - Fix for the partition changes (Christoph, Ming)

   - Fix for turning off iopoll with polled IO inflight (Jeffle)

   - nbd disconnect fix (Josef)

   - loop fsync error fix (Mauricio)

   - kyber update depth fix (Yang)

   - max_sectors alignment fix (Mikulas)

   - Add bio_max_segs helper (Matthew)"

* tag 'block-5.12-2021-02-27' of git://git.kernel.dk/linux-block: (21 commits)
  block: Add bio_max_segs
  blktrace: fix documentation for blk_fill_rw()
  block: memory allocations in bounce_clone_bio must not fail
  block: remove the gfp_mask argument to bounce_clone_bio
  block: fix bounce_clone_bio for passthrough bios
  block-crypto-fallback: use a bio_set for splitting bios
  block: fix logging on capacity change
  blk-settings: align max_sectors on "logical_block_size" boundary
  block: reopen the device in blkdev_reread_part
  block: don't skip empty device in in disk_uevent
  blktrace: remove debugfs file dentries from struct blk_trace
  nbd: handle device refs for DESTROY_ON_DISCONNECT properly
  kyber: introduce kyber_depth_updated()
  loop: fix I/O error on fsync() in detached loop devices
  block: fix potential IO hang when turning off io_poll
  block: get rid of the trace rq insert wrapper
  blktrace: fix blk_rq_merge documentation
  blktrace: fix blk_rq_issue documentation
  blktrace: add blk_fill_rwbs documentation comment
  block: remove superfluous param in blk_fill_rwbs()
  ...

Commit 78d3bb44 ("kbuild: Fix <linux/version.h> for empty SUBLEVEL
or PATCHLEVEL") fixed the build error for empty SUBLEVEL or PATCHLEVEL
by prepending a zero.

Commit 9b82f13e ("kbuild: clamp SUBLEVEL to 255") re-introduced
this issue.

This time, we cannot take the same approach because we have C code:

  #define LINUX_VERSION_PATCHLEVEL $(PATCHLEVEL)
  #define LINUX_VERSION_SUBLEVEL $(SUBLEVEL)

Replace empty SUBLEVEL/PATCHLEVEL with a zero.

Fixes: 9b82f13e ("kbuild: clamp SUBLEVEL to 255")
Reported-by: Christian Zigotzky <chzigotzky@xenosoft.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-and-tested-by: Sasha Levin <sashal@kernel.org>

'make -s' should be really silent. However, 'make -s V=1' prints noisy
log messages from some shell scripts.

Of course, such a combination is odd, but the build system needs to do
the right thing even if a user gives strange input.

If -s is given, KBUILD_VERBOSE should be forced to 0.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

READELF is defined by the top Makefile.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Commit cd195bc4 ("kbuild: split adjust_autoksyms.sh in two parts")
split out the code that needs include/config/auto.conf.

This script no longer needs to include include/config/auto.conf.

Fixes: cd195bc4 ("kbuild: split adjust_autoksyms.sh in two parts")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

Commit fbe078d3 ("kbuild: lto: add a default list of used symbols")
does not work as expected if the .config file has already specified
CONFIG_UNUSED_KSYMS_WHITELIST="my/own/white/list" before enabling
CONFIG_LTO_CLANG.

So, the user-supplied whitelist and LTO-specific white list must be
independent of each other.

I refactored the shell script so CONFIG_MODVERSIONS and CONFIG_CLANG_LTO
handle whitelists in the same way.

Fixes: fbe078d3 ("kbuild: lto: add a default list of used symbols")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>

Pull io_uring thread rewrite from Jens Axboe:
 "This converts the io-wq workers to be forked off the tasks in question
  instead of being kernel threads that assume various bits of the
  original task identity.

  This kills > 400 lines of code from io_uring/io-wq, and it's the worst
  part of the code. We've had several bugs in this area, and the worry
  is always that we could be missing some pieces for file types doing
  unusual things (recent /dev/tty example comes to mind, userfaultfd
  reads installing file descriptors is another fun one... - both of
  which need special handling, and I bet it's not the last weird oddity
  we'll find).

  With these identical workers, we can have full confidence that we're
  never missing anything. That, in itself, is a huge win. Outside of
  that, it's also more efficient since we're not wasting space and code
  on tracking state, or switching between different states.

  I'm sure we're going to find little things to patch up after this
  series, but testing has been pretty thorough, from the usual
  regression suite to production. Any issue that may crop up should be
  manageable.

  There's also a nice series of further reductions we can do on top of
  this, but I wanted to get the meat of it out sooner rather than later.
  The general worry here isn't that it's fundamentally broken. Most of
  the little issues we've found over the last week have been related to
  just changes in how thread startup/exit is done, since that's the main
  difference between using kthreads and these kinds of threads. In fact,
  if all goes according to plan, I want to get this into the 5.10 and
  5.11 stable branches as well.

  That said, the changes outside of io_uring/io-wq are:

   - arch setup, simple one-liner to each arch copy_thread()
     implementation.

   - Removal of net and proc restrictions for io_uring, they are no
     longer needed or useful"

* tag 'io_uring-worker.v3-2021-02-25' of git://git.kernel.dk/linux-block: (30 commits)
  io-wq: remove now unused IO_WQ_BIT_ERROR
  io_uring: fix SQPOLL thread handling over exec
  io-wq: improve manager/worker handling over exec
  io_uring: ensure SQPOLL startup is triggered before error shutdown
  io-wq: make buffered file write hashed work map per-ctx
  io-wq: fix race around io_worker grabbing
  io-wq: fix races around manager/worker creation and task exit
  io_uring: ensure io-wq context is always destroyed for tasks
  arch: ensure parisc/powerpc handle PF_IO_WORKER in copy_thread()
  io_uring: cleanup ->user usage
  io-wq: remove nr_process accounting
  io_uring: flag new native workers with IORING_FEAT_NATIVE_WORKERS
  net: remove cmsg restriction from io_uring based send/recvmsg calls
  Revert "proc: don't allow async path resolution of /proc/self components"
  Revert "proc: don't allow async path resolution of /proc/thread-self components"
  io_uring: move SQPOLL thread io-wq forked worker
  io-wq: make io_wq_fork_thread() available to other users
  io-wq: only remove worker from free_list, if it was there
  io_uring: remove io_identity
  io_uring: remove any grabbing of context
  ...

Pull misc vfs updates from Al Viro:
 "Assorted stuff pile - no common topic here"

* 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs:
  whack-a-mole: don't open-code iminor/imajor
  9p: fix misuse of sscanf() in v9fs_stat2inode()
  audit_alloc_mark(): don't open-code ERR_CAST()
  fs/inode.c: make inode_init_always() initialize i_ino to 0
  vfs: don't unnecessarily clone write access for writable fds

Pull i2c fixes from Wolfram Sang:
 "Three more bugfixes and one revert. I accidently applied one patch too
  early"

* 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux:
  i2c: exynos5: Preserve high speed master code
  Revert "i2c: i2c-qcom-geni: Add shutdown callback for i2c"
  i2c: designware: Get right data length
  i2c: brcmstb: Fix brcmstd_send_i2c_cmd condition

: error: C++ style comments are not allowed in ISO C90
 // Copyright (C) 2018 Hangzhou C-SKY Microsystems co.,ltd.
 ^
error: (this will be reported only once per input file)
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>

The existing code is essentially
free_initmem_default()->free_reserved_area() without poisoning.

Note that existing code missed to update the managed page count of the
zone.

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Wei Yang <richard.weiyang@linux.alibaba.com>
Tested-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: David Hildenbrand <david@redhat.com>

The max_mapnr is the number of PFNs, not absolute PFN offset.
Using set_max_mapnr API instead of setting the value directly.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>

It could help to reduce the latency of the time-related functions
in user space.

We have referenced arm's and riscv's implementation for the patch.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Vincent Chen <vincent.chen@sifive.com>
Cc: Arnd Bergmann <arnd@arndb.de>

The type of 'val' is 'unsigned long' in simulate_blz32, so 'val < 0'
can't be true.

Cast 'val' to 'long' here to determine branch token or not,

Fixup instructions: bnezad32, bhsz32, bhz32, blsz32, blz32

Link: https://lore.kernel.org/linux-csky/CAJF2gTQjKXR9gpo06WAWG1aquiT87mATiMGorXs6ChxOxoe90Q@mail.gmail.com/T/#tSigned-off-by: Guo Ren <guoren@linux.alibaba.com>
Co-developed-by: Menglong Dong <dong.menglong@zte.com.cn>
Signed-off-by: Menglong Dong <dong.menglong@zte.com.cn>

Current csky's swappon is broken by wrong swap PTE entry format.
Now redesign the new format for abiv1 & abiv2 and make swappon +
zram work properly on csky machines.

C-SKY PTE has VALID, DIRTY to emulate PRESENT, READ, WRITE, EXEC
attributes. GLOBAL bit is shared by two pages in the same tlb
entry. So we need to keep GLOBAL, VALID, PRESENT zero in swp_pte.

To distinguish PAGE_NONE and swp_pte, we need to use an additional
bit (abiv1 is _PAGE_READ, abiv2 is _PAGE_WRITE).
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Arnd Bergmann <arnd@arndb.de>

C-SKY page table attributes only have 'Dirty' and 'Valid' to
emulate 'PRESENT, READ, WRITE, EXEC, DIRTY, ACCESSED'.

This patch cleanup unnecessary definition.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Cc: Arnd Bergmann <arnd@arndb.de>

Some randconfig builds fail with undefined references to _mcount
when CONFIG_TRIM_UNUSED_KSYMS is set:

ERROR: modpost: "_mcount" [drivers/tee/optee/optee.ko] undefined!
ERROR: modpost: "_mcount" [drivers/fsi/fsi-occ.ko] undefined!
ERROR: modpost: "_mcount" [drivers/fpga/dfl-pci.ko] undefined!

Since there is already a list of symbols that get generated at link
time, add this one as well.

Fixes: fbe078d3 ("kbuild: lto: add a default list of used symbols")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

After the following patches,

  commit de043da0 ("RISC-V: Fix usage of memblock_enforce_memory_limit")
  commit 1bd14a66 ("RISC-V: Remove any memblock representing unusable memory area")
  commit b10d6bca ("arch, drivers: replace for_each_membock() with for_each_mem_range()")

some logic is useless, kill the mem_start/start/end and unneeded code.
Reviewed-by: Atish Patra <atish.patra@wdc.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>

The CPU hotplug support has been tested on QEMU, Spike, and SiFive
Unleashed so let's enable it by default in RV32 and RV64 defconfigs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>

In theory these are orthogonal, but in practice all NUMA systems are
SMP.  NUMA && !SMP doesn't build, everyone else is coupling them, and I
don't really see any value in supporting that configuration.

Fixes: 4f0e8eef ("riscv: Add numa support for riscv64 platform")
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Atish Patra <atishp@atishpatra.org>
Reported-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Tested-by:  Kefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>