1. 30 Aug, 2013 4 commits
    • Dave Chinner's avatar
      xfs: inode buffers may not be valid during recovery readahead · d8914002
      Dave Chinner authored
      CRC enabled filesystems fail log recovery with 100% reliability on
      xfstests xfs/085 with the following failure:
      
      XFS (vdb): Mounting Filesystem
      XFS (vdb): Starting recovery (logdev: internal)
      XFS (vdb): Corruption detected. Unmount and run xfs_repair
      XFS (vdb): bad inode magic/vsn daddr 144 #0 (magic=0)
      XFS: Assertion failed: 0, file: fs/xfs/xfs_inode_buf.c, line: 95
      
      The problem is that the inode buffer has not been recovered before
      the readahead on the inode buffer is issued. The checkpoint being
      recovered actually allocates the inode chunk we are doing readahead
      from, so what comes from disk during readahead is essentially
      random and the verifier barfs on it.
      
      This inode buffer readahead problem affects non-crc filesystems,
      too, but xfstests does not trigger it at all on such
      configurations....
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      d8914002
    • Dave Chinner's avatar
      xfs: check LSN ordering for v5 superblocks during recovery · 50d5c8d8
      Dave Chinner authored
      Log recovery has some strict ordering requirements which unordered
      or reordered metadata writeback can defeat. This can occur when an
      item is logged in a transaction, written back to disk, and then
      logged in a new transaction before the tail of the log is moved past
      the original modification.
      
      The result of this is that when we read an object off disk for
      recovery purposes, the buffer that we read may not contain the
      object type that recovery is expecting and hence at the end of the
      checkpoint being recovered we have an invalid object in memory.
      
      This isn't usually a problem, as recovery will then replay all the
      other checkpoints and that brings the object back to a valid and
      correct state, but the issue is that while the object is in the
      invalid state it can be flushed to disk. This results in the object
      verifier failing and triggering a corruption shutdown of log
      recover. This is correct behaviour for the verifiers - the problem
      is that we are not detecting that the object we've read off disk is
      newer than the transaction we are replaying.
      
      All metadata in v5 filesystems has the LSN of it's last modification
      stamped in it. This enabled log recover to read that field and
      determine the age of the object on disk correctly. If the LSN of the
      object on disk is older than the transaction being replayed, then we
      replay the modification. If the LSN of the object matches or is more
      recent than the transaction's LSN, then we should avoid overwriting
      the object as that is what leads to the transient corrupt state.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      50d5c8d8
    • Dave Chinner's avatar
      xfs: btree block LSN escaping to disk uninitialised · b58fa554
      Dave Chinner authored
      When testing LSN ordering code for v5 superblocks, it was discovered
      that the the LSN embedded in the generic btree blocks was
      occasionally uninitialised. These values didn't get written to disk
      by metadata writeback - they got written by previous transactions in
      log recovery.
      
      The issue is here that the when the block is first allocated and
      initialised, the LSN field was not initialised - it gets overwritten
      before IO is issued on the buffer - but the value that is logged by
      transactions that modify the header before it is written to disk
      (and initialised) contain garbage. Hence the first recovery of the
      buffer will stamp garbage into the LSN field, and that can cause
      subsequent transactions to not replay correctly.
      
      The fix is simply to initialise the bb_lsn field to zero when we
      initialise the block for the first time.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      b58fa554
    • Dave Chinner's avatar
      XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file:... · 37804376
      Dave Chinner authored
      XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568
      
      The calculation doesn't take into account the size of the dir v3
      header, so overestimates the hash entries in a node. This causes
      directory buffer overruns when splitting and merging nodes.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Tested-by: default avatarBrian Foster <bfoster@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      
      37804376
  2. 29 Aug, 2013 2 commits
    • Dave Chinner's avatar
      xfs: fix bad dquot buffer size in log recovery readahead · 0f0d3345
      Dave Chinner authored
      xfstests xfs/087 fails 100% reliably with this assert:
      
      XFS (vdb): Mounting Filesystem
      XFS (vdb): Starting recovery (logdev: internal)
      XFS: Assertion failed: bp->b_flags & XBF_STALE, file: fs/xfs/xfs_buf.c, line: 548
      
      while trying to read a dquot buffer in xlog_recover_dquot_ra_pass2().
      
      The issue is that the buffer length to read that is passed to
      xfs_buf_readahead is in units of filesystem blocks, not disk blocks.
      (i.e. FSB, not daddr). Fix it but putting the correct conversion in
      place.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      0f0d3345
    • Dave Chinner's avatar
      xfs: don't account buffer cancellation during log recovery readahead · 84a5b730
      Dave Chinner authored
      When doing readhaead in log recovery, we check to see if buffers are
      cancelled before doing readahead. If we find a cancelled buffer,
      however, we always decrement the reference count we have on it, and
      that means that readahead is causing a double decrement of the
      cancelled buffer reference count.
      
      This results in log recovery *replaying cancelled buffers* as the
      actual recovery pass does not find the cancelled buffer entry in the
      commit phase of the second pass across a transaction. On debug
      kernels, this results in an ASSERT failure like so:
      
      XFS: Assertion failed: !(flags & XFS_BLF_CANCEL), file: fs/xfs/xfs_log_recover.c, line: 1815
      
      xfstests generic/311 reproduces this ASSERT failure with 100%
      reproducability.
      
      Fix it by making readahead only peek at the buffer cancelled state
      rather than the full accounting that xlog_check_buffer_cancelled()
      does.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      84a5b730
  3. 26 Aug, 2013 2 commits
  4. 23 Aug, 2013 2 commits
  5. 22 Aug, 2013 4 commits
    • Richard Weinberger's avatar
      xfs: Register hotcpu notifier after initialization · 46677e67
      Richard Weinberger authored
      Currently the code initializizes mp->m_icsb_mutex and other things
      _after_ register_hotcpu_notifier().
      As the notifier takes mp->m_icsb_mutex it can happen
      that it takes the lock before it's initialization.
      Signed-off-by: default avatarRichard Weinberger <richard@nod.at>
      Reviewed-by: default avatarBen Myers <bpm@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      46677e67
    • Mark Tinguely's avatar
      xfs: add xfs sb v4 support for dirent filetype field · 3e3c51ce
      Mark Tinguely authored
      Add XFS superblock v4 support for the file type field in the
      directory entry feature.
      
      This support adds a feature bit for version 4 superblocks and
      leaves the original superblock 5 incompatibility bit.
      Signed-off-by: default avatarMark Tinguely <tinguely@sgi.com>
      Reviewed-by: default avatarGeoffrey Wehrman <gwehrman@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      3e3c51ce
    • Dave Chinner's avatar
      xfs: Add write support for dirent filetype field · 1c55cece
      Dave Chinner authored
      Add support to propagate and add filetype values into the on-disk
      directs. This involves passing the filetype into the xfs_da_args
      structure along with the name and namelength for direct operations,
      and encoding it into the dirent at the same time we write the inode
      number into the dirent.
      
      With write support, add the feature flag to the
      XFS_SB_FEAT_INCOMPAT_ALL mask so we can now mount filesystems with
      this feature set.
      
      Performance of directory recursion is now much improved. Parallel
      walk of ~50 million directory entries across hundreds of directories
      improves significantly. Unpatched, no CRCs:
      
      Walking via ls -R
      
      real    3m19.886s
      user    6m36.960s
      sys     28m19.087s
      
      THis is doing roughly 500 getdents() calls per second, and 250,000
      inode lookups per second to determine the inode type at roughly
      17,000 read IOPS. CPU usage is 90% kernel space.
      
      With dtype support patched in and the fileset recreated with CRCs
      enabled:
      
      Walking via ls -R
      
      real    0m31.316s
      user    6m32.975s
      sys     0m21.111s
      
      This is doing roughly 3500 getdents() calls per second at 16,000
      IOPS. There are no inode lookups at all. CPU usages is almost 100%
      userspace.
      
      This is a big win for recursive directory walks that only need to
      find file names and file types.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      1c55cece
    • Dave Chinner's avatar
      xfs: Add read-only support for dirent filetype field · 0cb97766
      Dave Chinner authored
      Add support for the file type field in directory entries so that
      readdir can return the type of the inode the dirent points to to
      userspace without first having to read the inode off disk.
      
      The encoding of the type field is a single byte that is added to the
      end of the directory entry name length. For all intents and
      purposes, it appends a "hidden" byte to the name field which
      contains the type information. As the directory entry is already of
      dynamic size, helpers are already required to access and decode the
      direct entry structures.
      
      Hence the relevent extraction and iteration helpers are updated to
      understand the hidden byte.  Helpers for reading and writing the
      filetype field from the directory entries are also added. Only the
      read helpers are used by this patch.  It also adds all the code
      necessary to read the type information out of the dirents on disk.
      
      Further we add the superblock feature bit and helpers to indicate
      that we understand the on-disk format change. This is not a
      compatible change - existing kernels cannot read the new format
      successfully - so an incompatible feature flag is added. We don't
      yet allow filesystems to mount with this flag yet - that will be
      added once write support is added.
      
      Finally, the code to take the type from the VFS, convert it to an
      XFS on-disk type and put it into the xfs_name structures passed
      around is added, but the directory code does not use this field yet.
      That will be in the next patch.
      Signed-off-by: default avatarDave Chinner <dchinner@redhat.com>
      Reviewed-by: default avatarMark Tinguely <tinguely@sgi.com>
      Signed-off-by: default avatarBen Myers <bpm@sgi.com>
      0cb97766
  6. 21 Aug, 2013 1 commit
  7. 20 Aug, 2013 25 commits