1. 08 Dec, 2008 1 commit
    • Chris Mason's avatar
      Btrfs: move data checksumming into a dedicated tree · d20f7043
      Chris Mason authored
      Btrfs stores checksums for each data block.  Until now, they have
      been stored in the subvolume trees, indexed by the inode that is
      referencing the data block.  This means that when we read the inode,
      we've probably read in at least some checksums as well.
      
      But, this has a few problems:
      
      * The checksums are indexed by logical offset in the file.  When
      compression is on, this means we have to do the expensive checksumming
      on the uncompressed data.  It would be faster if we could checksum
      the compressed data instead.
      
      * If we implement encryption, we'll be checksumming the plain text and
      storing that on disk.  This is significantly less secure.
      
      * For either compression or encryption, we have to get the plain text
      back before we can verify the checksum as correct.  This makes the raid
      layer balancing and extent moving much more expensive.
      
      * It makes the front end caching code more complex, as we have touch
      the subvolume and inodes as we cache extents.
      
      * There is potentitally one copy of the checksum in each subvolume
      referencing an extent.
      
      The solution used here is to store the extent checksums in a dedicated
      tree.  This allows us to index the checksums by phyiscal extent
      start and length.  It means:
      
      * The checksum is against the data stored on disk, after any compression
      or encryption is done.
      
      * The checksum is stored in a central location, and can be verified without
      following back references, or reading inodes.
      
      This makes compression significantly faster by reducing the amount of
      data that needs to be checksummed.  It will also allow much faster
      raid management code in general.
      
      The checksums are indexed by a key with a fixed objectid (a magic value
      in ctree.h) and offset set to the starting byte of the extent.  This
      allows us to copy the checksum items into the fsync log tree directly (or
      any other tree), without having to invent a second format for them.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      d20f7043
  2. 02 Dec, 2008 15 commits
  3. 20 Nov, 2008 7 commits
  4. 19 Nov, 2008 4 commits
    • Chris Mason's avatar
    • Josef Bacik's avatar
      Btrfs: fix free space accounting when unpinning extents · 07103a3c
      Josef Bacik authored
      This patch fixes what I hope is the last early ENOSPC bug left.  I did not know
      that pinned extents would merge into one big extent when inserted on to the
      pinned extent tree, so I was adding free space to a block group that could
      possibly span multiple block groups.
      
      This is a big issue because first that space doesn't exist in that block group,
      and second we won't actually use that space because there are a bunch of other
      checks to make sure we're allocating within the constraints of the block group.
      
      This patch fixes the problem by adding the btrfs_add_free_space to
      btrfs_update_pinned_extents which makes sure we are adding the appropriate
      amount of free space to the appropriate block group.  Thanks much to Lee Trager
      for running my myriad of debug patches to help me track this problem down.
      Thank you,
      Signed-off-by: default avatarJosef Bacik <jbacik@redhat.com>
      07103a3c
    • Chris Mason's avatar
      Btrfs: Do fsync log replay when mount -o ro, except when on readonly media · 7c2ca468
      Chris Mason authored
      fsync log replay can change the filesystem, so it cannot be delayed until
      mount -o rw,remount, and it can't be forgotten entirely.  So, this patch
      changes btrfs to do with reiserfs, ext3 and xfs do, which is to do the
      log replay even when mounted readonly.
      
      On a readonly device if log replay is required, the mount is aborted.
      
      Getting all of this right had required fixing up some of the error
      handling in open_ctree.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      7c2ca468
    • Chris Mason's avatar
      Btrfs: Avoid writeback stalls · d2c3f4f6
      Chris Mason authored
      While building large bios in writepages, btrfs may end up waiting
      for other page writeback to finish if WB_SYNC_ALL is used.
      
      While it is waiting, the bio it is building has a number of pages with the
      writeback bit set and they aren't getting to the disk any time soon.  This
      lowers the latencies of writeback in general by sending down the bio being
      built before waiting for other pages.
      
      The bio submission code tries to limit the total number of async bios in
      flight by waiting when we're over a certain number of async bios.  But,
      the waits are happening while writepages is building bios, and this can easily
      lead to stalls and other problems for people calling wait_on_page_writeback.
      
      The current fix is to let the congestion tests take care of waiting.
      
      sync() and others make sure to drain the current async requests to make
      sure that everything that was pending when the sync was started really get
      to disk.  The code would drain pending requests both before and after
      submitting a new request.
      
      But, if one of the requests is waiting for page writeback to finish,
      the draining waits might block that page writeback.  This changes the
      draining code to only wait after submitting the bio being processed.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      d2c3f4f6
  5. 18 Nov, 2008 10 commits
    • Chris Mason's avatar
      Btrfs: switch back to wait_on_page_writeback to wait on metadata writes · 105d931d
      Chris Mason authored
      The extent based waiting was using more CPU, and other fixes have helped
      with the unplug storm problems.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      105d931d
    • Chris Mason's avatar
    • Chris Mason's avatar
      Btrfs: unplug all devices in the unplug call back · 9f0ba5bd
      Chris Mason authored
      For larger multi-device filesystems, there was logic to limit the
      number of devices unplugged to just the page that was sent to our sync_page
      function.
      
      But, the code wasn't always unplugging the right device.  Since this was
      just an optimization, disable it for now.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      9f0ba5bd
    • Liu Hui's avatar
      Btrfs: Some fixes for batching extent insert. · b4eec2ca
      Liu Hui authored
      In insert_extents(), when ret==1 and last is not zero, it should
      check if the current inserted item is the last item in this batching
      inserts. If so, it should just break from loop. If not, 'cur =
      insert_list->next' will make no sense because the list is empty now,
      and 'op' will point to an unexpectable place.
      
      There are also some trivial fixs in this patch including one comment
      typo error and deleting two redundant lines.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      b4eec2ca
    • Chris Mason's avatar
      Btrfs: prevent loops in the directory tree when creating snapshots · ea9e8b11
      Chris Mason authored
      For a directory tree:
      
      /mnt/subvolA/subvolB
      
      btrfsctl -s /mnt/subvolA/subvolB /mnt
      
      Will create a directory loop with subvolA under subvolB.  This
      commit uses the forward refs for each subvol and snapshot to error out
      before creating the loop.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      ea9e8b11
    • Chris Mason's avatar
      Btrfs: Add backrefs and forward refs for subvols and snapshots · 0660b5af
      Chris Mason authored
      Subvols and snapshots can now be referenced from any point in the directory
      tree.  We need to maintain back refs for them so we can find lost
      subvols.
      
      Forward refs are added so that we know all of the subvols and
      snapshots referenced anywhere in the directory tree of a single subvol.  This
      can be used to do recursive snapshotting (but they aren't yet) and it is
      also used to detect and prevent directory loops when creating new snapshots.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      0660b5af
    • Chris Mason's avatar
      Btrfs: Give each subvol and snapshot their own anonymous devid · 3394e160
      Chris Mason authored
      Each subvolume has its own private inode number space, and so we need
      to fill in different device numbers for each subvolume to avoid confusing
      applications.
      
      This commit puts a struct super_block into struct btrfs_root so it can
      call set_anon_super() and get a different device number generated for
      each root.
      
      btrfs_rename is changed to prevent renames across subvols.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      3394e160
    • Chris Mason's avatar
      Btrfs: Allow subvolumes and snapshots anywhere in the directory tree · 3de4586c
      Chris Mason authored
      Before, all snapshots and subvolumes lived in a single flat directory.  This
      was awkward and confusing because the single flat directory was only writable
      with the ioctls.
      
      This commit changes the ioctls to create subvols and snapshots at any
      point in the directory tree.  This requires making separate ioctls for
      snapshot and subvol creation instead of a combining them into one.
      
      The subvol ioctl does:
      
      btrfsctl -S subvol_name parent_dir
      
      After the ioctl is done subvol_name lives inside parent_dir.
      
      The snapshot ioctl does:
      
      btrfsctl -s path_for_snapshot root_to_snapshot
      
      path_for_snapshot can be an absolute or relative path.  btrfsctl breaks it up
      into directory and basename components.
      
      root_to_snapshot can be any file or directory in the FS.  The snapshot
      is taken of the entire root where that file lives.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      3de4586c
    • Josef Bacik's avatar
      Btrfs: Add some debugging around the ENOSPC bugs · 4ce4cb52
      Josef Bacik authored
      Some people are still reporting problems with early enospc.  This
      will help narrown down the cause.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      4ce4cb52
    • Josef Bacik's avatar
      Btrfs: fix free space leak · e3e469f8
      Josef Bacik authored
      In my batch delete/update/insert patch I introduced a free space leak.  The
      extent that we do the original search on in free_extents is never pinned, so we
      always update the block saying that it has free space, but the free space never
      actually gets added to the free space tree, since op->del will always be 0 and
      it's never actually added to the pinned extents tree.
      
      This patch fixes this problem by making sure we call pin_down_bytes on the
      pending extent op and set op->del to the return value of pin_down_bytes so
      update_block_group is called with the right value.  This seems to fix the case
      where we were getting ENOSPC when there was plenty of space available.
      Signed-off-by: default avatarJosef Bacik <jbacik@redhat.com>
      e3e469f8
  6. 15 Nov, 2008 3 commits
    • Linus Torvalds's avatar
      Linux 2.6.28-rc5 · 9bf1a244
      Linus Torvalds authored
      9bf1a244
    • Al Viro's avatar
      Fix inotify watch removal/umount races · 8f7b0ba1
      Al Viro authored
      Inotify watch removals suck violently.
      
      To kick the watch out we need (in this order) inode->inotify_mutex and
      ih->mutex.  That's fine if we have a hold on inode; however, for all
      other cases we need to make damn sure we don't race with umount.  We can
      *NOT* just grab a reference to a watch - inotify_unmount_inodes() will
      happily sail past it and we'll end with reference to inode potentially
      outliving its superblock.
      
      Ideally we just want to grab an active reference to superblock if we
      can; that will make sure we won't go into inotify_umount_inodes() until
      we are done.  Cleanup is just deactivate_super().
      
      However, that leaves a messy case - what if we *are* racing with
      umount() and active references to superblock can't be acquired anymore?
      We can bump ->s_count, grab ->s_umount, which will almost certainly wait
      until the superblock is shut down and the watch in question is pining
      for fjords.  That's fine, but there is a problem - we might have hit the
      window between ->s_active getting to 0 / ->s_count - below S_BIAS (i.e.
      the moment when superblock is past the point of no return and is heading
      for shutdown) and the moment when deactivate_super() acquires
      ->s_umount.
      
      We could just do drop_super() yield() and retry, but that's rather
      antisocial and this stuff is luser-triggerable.  OTOH, having grabbed
      ->s_umount and having found that we'd got there first (i.e.  that
      ->s_root is non-NULL) we know that we won't race with
      inotify_umount_inodes().
      
      So we could grab a reference to watch and do the rest as above, just
      with drop_super() instead of deactivate_super(), right? Wrong.  We had
      to drop ih->mutex before we could grab ->s_umount.  So the watch
      could've been gone already.
      
      That still can be dealt with - we need to save watch->wd, do idr_find()
      and compare its result with our pointer.  If they match, we either have
      the damn thing still alive or we'd lost not one but two races at once,
      the watch had been killed and a new one got created with the same ->wd
      at the same address.  That couldn't have happened in inotify_destroy(),
      but inotify_rm_wd() could run into that.  Still, "new one got created"
      is not a problem - we have every right to kill it or leave it alone,
      whatever's more convenient.
      
      So we can use idr_find(...) == watch && watch->inode->i_sb == sb as
      "grab it and kill it" check.  If it's been our original watch, we are
      fine, if it's a newcomer - nevermind, just pretend that we'd won the
      race and kill the fscker anyway; we are safe since we know that its
      superblock won't be going away.
      
      And yes, this is far beyond mere "not very pretty"; so's the entire
      concept of inotify to start with.
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      Acked-by: default avatarGreg KH <greg@kroah.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      8f7b0ba1
    • Huang Weiyi's avatar
      LIS3LV02Dx: remove unused #include <version.h> · 0d3b7100
      Huang Weiyi authored
      The file(s) below do not use LINUX_VERSION_CODE nor KERNEL_VERSION.
        drivers/hwmon/lis3lv02d.c
      
      This patch removes the said #include <version.h>.
      Signed-off-by: default avatarHuang Weiyi <weiyi.huang@gmail.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      0d3b7100