1. 29 Nov, 2011 1 commit
    • Mike Marciniszyn's avatar
      IB/ipoib: Prevent hung task or softlockup processing multicast response · 3874397c
      Mike Marciniszyn authored
      This following can occur with ipoib when processing a multicast reponse:
      
          BUG: soft lockup - CPU#0 stuck for 67s! [ib_mad1:982]
          Modules linked in: ...
          CPU 0:
          Modules linked in: ...
          Pid: 982, comm: ib_mad1 Not tainted 2.6.32-131.0.15.el6.x86_64 #1 ProLiant DL160 G5
          RIP: 0010:[<ffffffff814ddb27>]  [<ffffffff814ddb27>] _spin_unlock_irqrestore+0x17/0x20
          RSP: 0018:ffff8802119ed860  EFLAGS: 00000246
          0000000000000004 RBX: ffff8802119ed860 RCX: 000000000000a299
          RDX: ffff88021086c700 RSI: 0000000000000246 RDI: 0000000000000246
          RBP: ffffffff8100bc8e R08: ffff880210ac229c R09: 0000000000000000
          R10: ffff88021278aab8 R11: 0000000000000000 R12: ffff8802119ed860
          R13: ffffffff8100be6e R14: 0000000000000001 R15: 0000000000000003
          FS:  0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
          CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
          CR2: 00000000006d4840 CR3: 0000000209aa5000 CR4: 00000000000406f0
          DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
          DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
          Call Trace:
          [<ffffffffa032c247>] ? ipoib_mcast_send+0x157/0x480 [ib_ipoib]
          [<ffffffff8100bc8e>] ? apic_timer_interrupt+0xe/0x20
          [<ffffffff8100bc8e>] ? apic_timer_interrupt+0xe/0x20
          [<ffffffffa03283d4>] ? ipoib_path_lookup+0x124/0x2d0 [ib_ipoib]
          [<ffffffffa03286fc>] ? ipoib_start_xmit+0x17c/0x430 [ib_ipoib]
          [<ffffffff8141e758>] ? dev_hard_start_xmit+0x2c8/0x3f0
          [<ffffffff81439d0a>] ? sch_direct_xmit+0x15a/0x1c0
          [<ffffffff81423098>] ? dev_queue_xmit+0x388/0x4d0
          [<ffffffffa032d6b7>] ? ipoib_mcast_join_finish+0x2c7/0x510 [ib_ipoib]
          [<ffffffffa032dab8>] ? ipoib_mcast_sendonly_join_complete+0x1b8/0x1f0 [ib_ipoib]
          [<ffffffffa02a0946>] ? mcast_work_handler+0x1a6/0x710 [ib_sa]
          [<ffffffffa015f01e>] ? ib_send_mad+0xfe/0x3c0 [ib_mad]
          [<ffffffffa00f6c93>] ? ib_get_cached_lmc+0xa3/0xb0 [ib_core]
          [<ffffffffa02a0f9b>] ? join_handler+0xeb/0x200 [ib_sa]
          [<ffffffffa029e4fc>] ? ib_sa_mcmember_rec_callback+0x5c/0xa0 [ib_sa]
          [<ffffffffa029e79c>] ? recv_handler+0x3c/0x70 [ib_sa]
          [<ffffffffa01603a4>] ? ib_mad_completion_handler+0x844/0x9d0 [ib_mad]
          [<ffffffffa015fb60>] ? ib_mad_completion_handler+0x0/0x9d0 [ib_mad]
          [<ffffffff81088830>] ? worker_thread+0x170/0x2a0
          [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
          [<ffffffff810886c0>] ? worker_thread+0x0/0x2a0
          [<ffffffff8108ddf6>] ? kthread+0x96/0xa0
          [<ffffffff8100c1ca>] ? child_rip+0xa/0x20
      
      Coinciding with stack trace is the following message:
      
          ib0: ib_address_create failed
      
      The code below in ipoib_mcast_join_finish() will note the above
      failure in the address handle but otherwise continue:
      
                      ah = ipoib_create_ah(dev, priv->pd, &av);
                      if (!ah) {
                              ipoib_warn(priv, "ib_address_create failed\n");
                      } else {
      
      The while loop at the bottom of ipoib_mcast_join_finish() will attempt
      to send queued multicast packets in mcast->pkt_queue and eventually
      end up in ipoib_mcast_send():
      
              if (!mcast->ah) {
                      if (skb_queue_len(&mcast->pkt_queue) < IPOIB_MAX_MCAST_QUEUE)
                              skb_queue_tail(&mcast->pkt_queue, skb);
                      else {
                              ++dev->stats.tx_dropped;
                              dev_kfree_skb_any(skb);
                      }
      
      My read is that the code will requeue the packet and return to the
      ipoib_mcast_join_finish() while loop and the stage is set for the
      "hung" task diagnostic as the while loop never sees a non-NULL ah, and
      will do nothing to resolve.
      
      There are GFP_ATOMIC allocates in the provider routines, so this is
      possible and should be dealt with.
      
      The test that induced the failure is associated with a host SM on the
      same server during a shutdown.
      
      This patch causes ipoib_mcast_join_finish() to exit with an error
      which will flush the queued mcast packets.  Nothing is done to unwind
      the QP attached state so that subsequent sends from above will retry
      the join.
      Reviewed-by: default avatarRam Vepa <ram.vepa@qlogic.com>
      Reviewed-by: default avatarGary Leshner <gary.leshner@qlogic.com>
      Signed-off-by: default avatarMike Marciniszyn <mike.marciniszyn@qlogic.com>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      3874397c
  2. 08 Nov, 2011 2 commits
    • Linus Torvalds's avatar
      Linux 3.2-rc1 · 1ea6b8f4
      Linus Torvalds authored
      .. with new name.  Because nothing says "really solid kernel release"
      like naming it after an extinct animal that just happened to be in the
      news lately.
      1ea6b8f4
    • Linus Torvalds's avatar
      Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap · 075cb105
      Linus Torvalds authored
      * 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap: (31 commits)
        ARM: OMAP: Fix export.h or module.h includes
        ARM: OMAP: omap_device: Include linux/export.h
        ARM: OMAP2: Fix H4 matrix keyboard warning
        ARM: OMAP1: Remove unused omap-alsa.h
        ARM: OMAP1: Fix warnings about enabling 32 KiHz timer
        ARM: OMAP2+: timer: Remove omap_device_pm_latency
        ARM: OMAP2+: clock data: Remove redundant timer clkdev
        ARM: OMAP: Devkit8000: Remove double omap_mux_init_gpio
        ARM: OMAP: usb: musb: OMAP: Delete unused function
        MAINTAINERS: Update linux-omap git repository
        ARM: OMAP: change get_context_loss_count ret value to int
        ARM: OMAP4: hsmmc: configure SDMMC1_DR0 properly
        ARM: OMAP4: hsmmc: Fix Pbias configuration on regulator OFF
        ARM: OMAP3: hwmod: fix variant registration and remove SmartReflex from common list
        ARM: OMAP: I2C: Fix omap_register_i2c_bus() return value on success
        ARM: OMAP: dmtimer: Include linux/module.h
        ARM: OMAP2+: l3-noc: Include linux/module.h
        ARM: OMAP2+: devices: Fixes for McPDM
        ARM: OMAP: Fix errors and warnings when building for one board
        ARM: OMAP3: PM: restrict erratum i443 handling to OMAP3430 only
        ...
      075cb105
  3. 07 Nov, 2011 37 commits