1. 02 Nov, 2017 8 commits
    • Mikulas Patocka's avatar
      md: use TASK_IDLE instead of blocking signals · ae89fd3d
      Mikulas Patocka authored
      Hi - I submit this patch for the next merge window:
      
      Some times ago, I made a patch f9c79bc0 that blocks signals around the
      schedule() calls in MD. The MD subsystem needs to do an uninterruptible
      sleep that is not accounted in load average - so we block signals and use
      interruptible sleep.
      
      The kernel has a special TASK_IDLE state for this purpose, so we can use
      it instead of blocking signals. This patch doesn't fix any bug, it just
      makes the code simpler.
      Signed-off-by: default avatarMikulas Patocka <mpatocka@redhat.com>
      Acked-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      ae89fd3d
    • NeilBrown's avatar
      md: remove special meaning of ->quiesce(.., 2) · b03e0ccb
      NeilBrown authored
      The '2' argument means "wake up anything that is waiting".
      This is an inelegant part of the design and was added
      to help support management of suspend_lo/suspend_hi setting.
      Now that suspend_lo/hi is managed in mddev_suspend/resume,
      that need is gone.
      These is still a couple of places where we call 'quiesce'
      with an argument of '2', but they can safely be changed to
      call ->quiesce(.., 1); ->quiesce(.., 0) which
      achieve the same result at the small cost of pausing IO
      briefly.
      
      This removes a small "optimization" from suspend_{hi,lo}_store,
      but it isn't clear that optimization served a useful purpose.
      The code now is a lot clearer.
      Suggested-by: default avatarShaohua Li <shli@kernel.org>
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      b03e0ccb
    • NeilBrown's avatar
      md: allow metadata update while suspending. · 35bfc521
      NeilBrown authored
      There are various deadlocks that can occur
      when a thread holds reconfig_mutex and calls
      ->quiesce(mddev, 1).
      As some write request block waiting for
      metadata to be updated (e.g. to record device
      failure), and as the md thread updates the metadata
      while the reconfig mutex is held, holding the mutex
      can stop write requests completing, and this prevents
      ->quiesce(mddev, 1) from completing.
      
      ->quiesce() is now usually called from mddev_suspend(),
      and it is always called with reconfig_mutex held.  So
      at this time it is safe for the thread to update metadata
      without explicitly taking the lock.
      
      So add 2 new flags, one which says the unlocked updates is
      allowed, and one which ways it is happening.  Then allow it
      while the quiesce completes, and then wait for it to finish.
      Reported-and-tested-by: default avatarXiao Ni <xni@redhat.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      35bfc521
    • NeilBrown's avatar
      md: use mddev_suspend/resume instead of ->quiesce() · 9e1cc0a5
      NeilBrown authored
      mddev_suspend() is a more general interface than
      calling ->quiesce() and is so more extensible.  A
      future patch will make use of this.
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      9e1cc0a5
    • NeilBrown's avatar
      md: move suspend_hi/lo handling into core md code · b3143b9a
      NeilBrown authored
      responding to ->suspend_lo and ->suspend_hi is similar
      to responding to ->suspended.  It is best to wait in
      the common core code without incrementing ->active_io.
      This allows mddev_suspend()/mddev_resume() to work while
      requests are waiting for suspend_lo/hi to change.
      This is will be important after a subsequent patch
      which uses mddev_suspend() to synchronize updating for
      suspend_lo/hi.
      
      So move the code for testing suspend_lo/hi out of raid1.c
      and raid5.c, and place it in md.c
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      b3143b9a
    • NeilBrown's avatar
      md: don't call bitmap_create() while array is quiesced. · 52a0d49d
      NeilBrown authored
      bitmap_create() allocates memory with GFP_KERNEL and
      so can wait for IO.
      If called while the array is quiesced, it could wait indefinitely
      for write out to the array - deadlock.
      So call bitmap_create() before quiescing the array.
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      52a0d49d
    • NeilBrown's avatar
      md: always hold reconfig_mutex when calling mddev_suspend() · 4d5324f7
      NeilBrown authored
      Most often mddev_suspend() is called with
      reconfig_mutex held.  Make this a requirement in
      preparation a subsequent patch.  Also require
      reconfig_mutex to be held for mddev_resume(),
      partly for symmetry and partly to guarantee
      no races with incr/decr of mddev->suspend.
      
      Taking the mutex in r5c_disable_writeback_async() is
      a little tricky as this is called from a work queue
      via log->disable_writeback_work, and flush_work()
      is called on that while holding ->reconfig_mutex.
      If the work item hasn't run before flush_work()
      is called, the work function will not be able to
      get the mutex.
      
      So we use mddev_trylock() inside the wait_event() call, and have that
      abort when conf->log is set to NULL, which happens before
      flush_work() is called.
      We wait in mddev->sb_wait and ensure this is woken
      when any of the conditions change.  This requires
      waking mddev->sb_wait in mddev_unlock().  This is only
      like to trigger extra wake_ups of threads that needn't
      be woken when metadata is being written, and that
      doesn't happen often enough that the cost would be
      noticeable.
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      4d5324f7
    • NeilBrown's avatar
      md: forbid a RAID5 from having both a bitmap and a journal. · 230b55fa
      NeilBrown authored
      Having both a bitmap and a journal is pointless.
      Attempting to do so can corrupt the bitmap if the journal
      replay happens before the bitmap is initialized.
      Rather than try to avoid this corruption, simply
      refuse to allow arrays with both a bitmap and a journal.
      So:
       - if raid5_run sees both are present, fail.
       - if adding a bitmap finds a journal is present, fail
       - if adding a journal finds a bitmap is present, fail.
      
      Cc: stable@vger.kernel.org (4.10+)
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Tested-by: default avatarJoshua Kinard <kumba@gentoo.org>
      Acked-by: default avatarJoshua Kinard <kumba@gentoo.org>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      230b55fa
  2. 19 Oct, 2017 1 commit
    • NeilBrown's avatar
      raid5: Set R5_Expanded on parity devices as well as data. · 235b6003
      NeilBrown authored
      When reshaping a fully degraded raid5/raid6 to a larger
      nubmer of devices, the new device(s) are not in-sync
      and so that can make the newly grown stripe appear to be
      "failed".
      To avoid this, we set the R5_Expanded flag to say "Even though
      this device is not fully in-sync, this block is safe so
      don't treat the device as failed for this stripe".
      This flag is set for data devices, not not for parity devices.
      
      Consequently, if you have a RAID6 with two devices that are partly
      recovered and a spare, and start a reshape to include the spare,
      then when the reshape gets past the point where the recovery was
      up to, it will think the stripes are failed and will get into
      an infinite loop, failing to make progress.
      
      So when contructing parity on an EXPAND_READY stripe,
      set R5_Expanded.
      Reported-by: default avatarCurt <lightspd@gmail.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      235b6003
  3. 17 Oct, 2017 8 commits
  4. 09 Oct, 2017 1 commit
    • Guoqing Jiang's avatar
      md: always set THREAD_WAKEUP and wake up wqueue if thread existed · d1d90147
      Guoqing Jiang authored
      Since commit 4ad23a97 ("MD: use per-cpu counter for writes_pending"),
      the wait_queue is only got invoked if THREAD_WAKEUP is not set previously.
      
      With above change, I can see process_metadata_update could always hang on
      the wait queue, because mddev->thread could stay on 'D' status and the
      THREAD_WAKEUP flag is not cleared since there are lots of place to wake up
      mddev->thread. Then deadlock happened as follows:
      
      linux175:~ # ps aux|grep md|grep D
      root    20117   0.0 0.0         0   0 ? D   03:45   0:00 [md0_raid1]
      root    20125   0.0 0.0         0   0 ? D   03:45   0:00 [md0_cluster_rec]
      linux175:~ # cat /proc/20117/stack
      [<ffffffffa0635604>] dlm_lock_sync+0x94/0xd0 [md_cluster]
      [<ffffffffa0635674>] lock_token+0x34/0xd0 [md_cluster]
      [<ffffffffa0635804>] metadata_update_start+0x64/0x110 [md_cluster]
      [<ffffffffa04d985b>] md_update_sb.part.58+0x9b/0x860 [md_mod]
      [<ffffffffa04da035>] md_update_sb+0x15/0x30 [md_mod]
      [<ffffffffa04dc066>] md_check_recovery+0x266/0x490 [md_mod]
      [<ffffffffa06450e2>] raid1d+0x42/0x810 [raid1]
      [<ffffffffa04d2252>] md_thread+0x122/0x150 [md_mod]
      [<ffffffff81091741>] kthread+0x101/0x140
      linux175:~ # cat /proc/20125/stack
      [<ffffffffa0636679>] recv_daemon+0x3f9/0x5c0 [md_cluster]
      [<ffffffffa04d2252>] md_thread+0x122/0x150 [md_mod]
      [<ffffffff81091741>] kthread+0x101/0x140
      
      So let's revert the part of code in the commit to resovle the problem since
      we can't get lots of benefits of previous change.
      
      Fixes: 4ad23a97 ("MD: use per-cpu counter for writes_pending")
      Signed-off-by: default avatarGuoqing Jiang <gqjiang@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      d1d90147
  5. 05 Oct, 2017 1 commit
    • NeilBrown's avatar
      md: fix deadlock error in recent patch. · d47c8ad2
      NeilBrown authored
      A recent patch aimed to cause md_write_start() to fail (rather than
      block) when the mddev was suspending, so as to avoid deadlocks.
      Unfortunately the test in wait_event() was wrong, and it didn't change
      behaviour at all.
      
      We wait_event() must wait until the metadata is written OR the array is
      suspending.
      
      Fixes: cc27b0c7 ("md: fix deadlock between mddev_suspend() and md_write_start()")
      Cc: stable@vger.kernel.org
      Reported-by: default avatarXiao Ni <xni@redhat.com>
      Signed-off-by: default avatarNeilBrown <neilb@suse.com>
      Signed-off-by: default avatarShaohua Li <shli@fb.com>
      d47c8ad2
  6. 01 Oct, 2017 9 commits
    • Linus Torvalds's avatar
      Linux 4.14-rc3 · 9e66317d
      Linus Torvalds authored
      9e66317d
    • Linus Torvalds's avatar
      Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 368f8998
      Linus Torvalds authored
      Pull x86 fixes from Thomas Gleixner:
       "This contains the following fixes and improvements:
      
         - Avoid dereferencing an unprotected VMA pointer in the fault signal
           generation code
      
         - Fix inline asm call constraints for GCC 4.4
      
         - Use existing register variable to retrieve the stack pointer
           instead of forcing the compiler to create another indirect access
           which results in excessive extra 'mov %rsp, %<dst>' instructions
      
         - Disable branch profiling for the memory encryption code to prevent
           an early boot crash
      
         - Fix a sparse warning caused by casting the __user annotation in
           __get_user_asm_u64() away
      
         - Fix an off by one error in the loop termination of the error patch
           in the x86 sysfs init code
      
         - Add missing CPU IDs to various Intel specific drivers to enable the
           functionality on recent hardware
      
         - More (init) constification in the numachip code"
      
      * 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        x86/asm: Use register variable to get stack pointer value
        x86/mm: Disable branch profiling in mem_encrypt.c
        x86/asm: Fix inline asm call constraints for GCC 4.4
        perf/x86/intel/uncore: Correct num_boxes for IIO and IRP
        perf/x86/intel/rapl: Add missing CPU IDs
        perf/x86/msr: Add missing CPU IDs
        perf/x86/intel/cstate: Add missing CPU IDs
        x86: Don't cast away the __user in __get_user_asm_u64()
        x86/sysfs: Fix off-by-one error in loop termination
        x86/mm: Fix fault error path using unsafe vma pointer
        x86/numachip: Add const and __initconst to numachip2_clockevent
      368f8998
    • Linus Torvalds's avatar
      Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · c42ed9f9
      Linus Torvalds authored
      Pull timer fixes from Thomas Gleixner:
       "This adds a new timer wheel function which is required for the
        conversion of the timer callback function from the 'unsigned long
        data' argument to 'struct timer_list *timer'. This conversion has two
        benefits:
      
         1) It makes struct timer_list smaller
      
         2) Many callers hand in a pointer to the timer or to the structure
            containing the timer, which happens via type casting both at setup
            and in the callback. This change gets rid of the typecasts.
      
        Once the conversion is complete, which is planned for 4.15, the old
        setup function and the intermediate typecast in the new setup function
        go away along with the data field in struct timer_list.
      
        Merging this now into mainline allows a smooth queueing of the actual
        conversion in the affected maintainer trees without creating
        dependencies"
      
      * 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        um/time: Fixup namespace collision
        timer: Prepare to change timer callback argument type
      c42ed9f9
    • Linus Torvalds's avatar
      Merge branch 'smp-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 82513545
      Linus Torvalds authored
      Pull smp/hotplug fixes from Thomas Gleixner:
       "This addresses the fallout of the new lockdep mechanism which covers
        completions in the CPU hotplug code.
      
        The lockdep splats are false positives, but there is no way to
        annotate that reliably. The solution is to split the completions for
        CPU up and down, which requires some reshuffling of the failure
        rollback handling as well"
      
      * 'smp-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        smp/hotplug: Hotplug state fail injection
        smp/hotplug: Differentiate the AP completion between up and down
        smp/hotplug: Differentiate the AP-work lockdep class between up and down
        smp/hotplug: Callback vs state-machine consistency
        smp/hotplug: Rewrite AP state machine core
        smp/hotplug: Allow external multi-instance rollback
        smp/hotplug: Add state diagram
      82513545
    • Linus Torvalds's avatar
      Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 7e103ace
      Linus Torvalds authored
      Pull scheduler fixes from Thomas Gleixner:
       "The scheduler pull request comes with the following updates:
      
         - Prevent a divide by zero issue by validating the input value of
           sysctl_sched_time_avg
      
         - Make task state printing consistent all over the place and have
           explicit state characters for IDLE and PARKED so they wont be
           displayed as 'D' state which confuses tools"
      
      * 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        sched/sysctl: Check user input value of sysctl_sched_time_avg
        sched/debug: Add explicit TASK_PARKED printing
        sched/debug: Ignore TASK_IDLE for SysRq-W
        sched/debug: Add explicit TASK_IDLE printing
        sched/tracing: Use common task-state helpers
        sched/tracing: Fix trace_sched_switch task-state printing
        sched/debug: Remove unused variable
        sched/debug: Convert TASK_state to hex
        sched/debug: Implement consistent task-state printing
      7e103ace
    • Linus Torvalds's avatar
      Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 1c6f705b
      Linus Torvalds authored
      Pull perf fixes from Thomas Gleixner:
      
       - Prevent a division by zero in the perf aux buffer handling
      
       - Sync kernel headers with perf tool headers
      
       - Fix a build failure in the syscalltbl code
      
       - Make the debug messages of perf report --call-graph work correctly
      
       - Make sure that all required perf files are in the MANIFEST for
         container builds
      
       - Fix the atrr.exclude kernel handling so it respects the
         perf_event_paranoid and the user permissions
      
       - Make perf test on s390x work correctly
      
      * 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        perf/aux: Only update ->aux_wakeup in non-overwrite mode
        perf test: Fix vmlinux failure on s390x part 2
        perf test: Fix vmlinux failure on s390x
        perf tools: Fix syscalltbl build failure
        perf report: Fix debug messages with --call-graph option
        perf evsel: Fix attr.exclude_kernel setting for default cycles:p
        tools include: Sync kernel ABI headers with tooling headers
        perf tools: Get all of tools/{arch,include}/ in the MANIFEST
      1c6f705b
    • Linus Torvalds's avatar
      Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 1de47f3c
      Linus Torvalds authored
      Pull  locking fixes from Thomas Gleixner:
       "Two fixes for locking:
      
         - Plug a hole the pi_stat->owner serialization which was changed
           recently and failed to fixup two usage sites.
      
         - Prevent reordering of the rwsem_has_spinner() check vs the
           decrement of rwsem count in up_write() which causes a missed
           wakeup"
      
      * 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        locking/rwsem-xadd: Fix missed wakeup due to reordering of load
        futex: Fix pi_state->owner serialization
      1de47f3c
    • Linus Torvalds's avatar
      Merge branch 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 3d9d62b9
      Linus Torvalds authored
      Pull irq fixes from Thomas Gleixner:
      
       - Add a missing NULL pointer check in free_irq()
      
       - Fix a memory leak/memory corruption in the generic irq chip
      
       - Add missing rcu annotations for radix tree access
      
       - Use ffs instead of fls when extracting data from a chip register in
         the MIPS GIC irq driver
      
       - Fix the unmasking of IPI interrupts in the MIPS GIC driver so they
         end up at the target CPU and not at CPU0
      
      * 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        irq/generic-chip: Don't replace domain's name
        irqdomain: Add __rcu annotations to radix tree accessors
        irqchip/mips-gic: Use effective affinity to unmask
        irqchip/mips-gic: Fix shifts to extract register fields
        genirq: Check __free_irq() return value for NULL
      3d9d62b9
    • Linus Torvalds's avatar
      Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip · 156069f8
      Linus Torvalds authored
      Pull objtool fixes from Thomas Gleixner:
       "Two small fixes for objtool:
      
         - Support frame pointer setup via 'lea (%rsp), %rbp' which was not
           yet supported and caused build warnings
      
         - Disable unreacahble warnings for GCC4.4 and older to avoid false
           positives caused by the compiler itself"
      
      * 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
        objtool: Support unoptimized frame pointer setup
        objtool: Skip unreachable warnings for GCC 4.4 and older
      156069f8
  7. 30 Sep, 2017 4 commits
  8. 29 Sep, 2017 8 commits