1. 11 Nov, 2011 11 commits
    • David Sterba's avatar
      btrfs: rename the option to nospace_cache · 8965593e
      David Sterba authored
      Rename no_space_cache option to nospace_cache to be more consistent with
      the rest, where the simple prefix 'no' is used to negate an option.
      
      The option has been introduced during the -rc1 cycle and there are has not been
      widely used, so it's safe.
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.cz>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      8965593e
    • Arne Jansen's avatar
      Btrfs: handle bio_add_page failure gracefully in scrub · 69f4cb52
      Arne Jansen authored
      Currently scrub fails with ENOMEM when bio_add_page fails. Unfortunately
      dm based targets accept only one page per bio, thus making scrub always
      fails. This patch just submits the current bio when an error is encountered
      and starts a new one.
      Signed-off-by: default avatarArne Jansen <sensille@gmx.net>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      69f4cb52
    • Miao Xie's avatar
      Btrfs: fix deadlock caused by the race between relocation · 62f30c54
      Miao Xie authored
      We can not do flushable reservation for the relocation when we create snapshot,
      because it may make the transaction commit task and the flush task wait for
      each other and the deadlock happens.
      Signed-off-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      62f30c54
    • Josef Bacik's avatar
      Btrfs: only map pages if we know we need them when reading the space cache · 2f120c05
      Josef Bacik authored
      People have been running into a warning when loading space cache because the
      page is already mapped when trying to read in a bitmap.  The way we read in
      entries and pages is kind of convoluted, so fix it so that io_ctl_read_entry
      maps the entries if it needs to, and if it hits the end of the page it simply
      unmaps the page.  That way we can unconditionally unmap the io_ctl before
      reading in the bitmap and we should stop hitting these warnings.  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      2f120c05
    • Miao Xie's avatar
      Btrfs: fix orphan backref nodes · 76b9e23d
      Miao Xie authored
      If the root node of a fs/file tree is in the block group that is
      being relocated, but the others are not in the other block groups.
      when we create a snapshot for this tree between the relocation tree
      creation ends and ->create_reloc_tree is set to 0, Btrfs will create
      some backref nodes that are the lowest nodes of the backrefs cache.
      But we forget to add them into ->leaves list of the backref cache
      and deal with them, and at last, they will triggered BUG_ON().
      
        kernel BUG at fs/btrfs/relocation.c:239!
      
      This patch fixes it by adding them into ->leaves list of backref cache.
      Signed-off-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      76b9e23d
    • Miao Xie's avatar
      Btrfs: Abstract similar code for btrfs_block_rsv_add{, _noflush} · 61b520a9
      Miao Xie authored
      btrfs_block_rsv_add{, _noflush}() have similar code, so abstract that code.
      Signed-off-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      61b520a9
    • Miao Xie's avatar
      Btrfs: fix unreleased path in btrfs_orphan_cleanup() · 3254c876
      Miao Xie authored
      When we did stress test for the space relocation, the deadlock happened.
      By debugging, We found it was caused by the carelessness that we forgot
      to unlock the read lock of the extent buffers in btrfs_orphan_cleanup()
      before we end the transaction handle, so the transaction commit task waited
      the task, which called btrfs_orphan_cleanup(), to unlock the extent buffer,
      but that task waited the commit task to end the transaction commit, and
      the deadlock happened. Fix it.
      Signed-ff-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      3254c876
    • Miao Xie's avatar
      Btrfs: fix no reserved space for writing out inode cache · ba38eb4d
      Miao Xie authored
      I-node cache forgets to reserve the space when writing out it. And when
      we do some stress test, such as synctest, it will trigger WARN_ON() in
      use_block_rsv().
      
      WARNING: at fs/btrfs/extent-tree.c:5718 btrfs_alloc_free_block+0xbf/0x281 [btrfs]()
      ...
      Call Trace:
       [<ffffffff8104df86>] warn_slowpath_common+0x80/0x98
       [<ffffffff8104dfb3>] warn_slowpath_null+0x15/0x17
       [<ffffffffa0369c60>] btrfs_alloc_free_block+0xbf/0x281 [btrfs]
       [<ffffffff810cbcb8>] ? __set_page_dirty_nobuffers+0xfe/0x108
       [<ffffffffa035c040>] __btrfs_cow_block+0x118/0x3b5 [btrfs]
       [<ffffffffa035c7ba>] btrfs_cow_block+0x103/0x14e [btrfs]
       [<ffffffffa035e4c4>] btrfs_search_slot+0x249/0x6a4 [btrfs]
       [<ffffffffa036d086>] btrfs_lookup_inode+0x2a/0x8a [btrfs]
       [<ffffffffa03788b7>] btrfs_update_inode+0xaa/0x141 [btrfs]
       [<ffffffffa036d7ec>] btrfs_save_ino_cache+0xea/0x202 [btrfs]
       [<ffffffffa03a761e>] ? btrfs_update_reloc_root+0x17e/0x197 [btrfs]
       [<ffffffffa0373867>] commit_fs_roots+0xaa/0x158 [btrfs]
       [<ffffffffa03746a6>] btrfs_commit_transaction+0x405/0x731 [btrfs]
       [<ffffffff810690df>] ? wake_up_bit+0x25/0x25
       [<ffffffffa039d652>] ? btrfs_log_dentry_safe+0x43/0x51 [btrfs]
       [<ffffffffa0381c5f>] btrfs_sync_file+0x16a/0x198 [btrfs]
       [<ffffffff81122806>] ? mntput+0x21/0x23
       [<ffffffff8112d150>] vfs_fsync_range+0x18/0x21
       [<ffffffff8112d170>] vfs_fsync+0x17/0x19
       [<ffffffff8112d316>] do_fsync+0x29/0x3e
       [<ffffffff8112d348>] sys_fsync+0xb/0xf
       [<ffffffff81468352>] system_call_fastpath+0x16/0x1b
      
      Sometimes it causes BUG_ON() in the reservation code of the delayed inode
      is triggered.
      
      So we must reserve enough space for inode cache.
      
      Note: If we can not reserve the enough space for inode cache, we will
      give up writing out it.
      Signed-off-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      ba38eb4d
    • Miao Xie's avatar
      Btrfs: fix nocow when deleting the item · 924cd8fb
      Miao Xie authored
      btrfs_previous_item() just search the b+ tree, do not COW the nodes or leaves,
      if we modify the result of it, the meta-data will be broken. fix it.
      Signed-off-by: default avatarMiao Xie <miaox@cn.fujitsu.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      924cd8fb
    • Chris Mason's avatar
    • Chris Mason's avatar
      Btrfs: tweak the delayed inode reservations again · 2115133f
      Chris Mason authored
      Josef sent along an incremental to the inode reservation
      code to make sure we try and fall back to directly updating
      the inode item if things go horribly wrong.
      
      This reworks that patch slightly, adding a fallback function
      that will always try to update the inode item directly without
      going through the delayed_inode code.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      2115133f
  2. 09 Nov, 2011 5 commits
  3. 08 Nov, 2011 2 commits
    • Josef Bacik's avatar
      Btrfs: fix our reservations for updating an inode when completing io · 7fd2ae21
      Josef Bacik authored
      People have been reporting ENOSPC crashes in finish_ordered_io.  This is because
      we try to steal from the delalloc block rsv to satisfy a reservation to update
      the inode.  The problem with this is we don't explicitly save space for updating
      the inode when doing delalloc.  This is kind of a problem and we've gotten away
      with this because way back when we just stole from the delalloc reserve without
      any questions, and this worked out fine because generally speaking the leaf had
      been modified either by the mtime update when we did the original write or
      because we just updated the leaf when we inserted the file extent item, only on
      rare occasions had the leaf not actually been modified, and that was still ok
      because we'd just use a block or two out of the over-reservation that is
      delalloc.
      
      Then came the delayed inode stuff.  This is amazing, except it wants a full
      reservation for updating the inode since it may do it at some point down the
      road after we've written the blocks and we have to recow everything again.  This
      worked out because the delayed inode stuff just stole from the global reserve,
      that is until recently when I changed that because it caused other problems.
      
      So here we are, we're doing everything right and being screwed for it.  So take
      an extra reservation for the inode at delalloc reservation time and carry it
      through the life of the delalloc reservation.  If we need it we can steal it in
      the delayed inode stuff.  If we have already stolen it try and do a normal
      metadata reservation.  If that fails try to steal from the delalloc reservation.
      If _that_ fails we'll get a WARN_ON() so I can start thinking of a better way to
      solve this and in the meantime we'll steal from the global reserve.
      
      With this patch I ran xfstests 13 in a loop for a couple of hours and didn't see
      any problems.
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      7fd2ae21
    • Chris Mason's avatar
      Btrfs: fix oops on NULL trans handle in btrfs_truncate · 917c16b2
      Chris Mason authored
      If we fail to reserve space in the transaction during truncate, we can
      error out with a NULL trans handle.  The cleanup code needs an extra
      check to make sure we aren't trying to use the bad handle.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      917c16b2
  4. 07 Nov, 2011 1 commit
    • slyich@gmail.com's avatar
      btrfs: fix double-free 'tree_root' in 'btrfs_mount()' · 45ea6095
      slyich@gmail.com authored
      On error path 'tree_root' is treed in 'free_fs_info()'.
      No need to free it explicitely. Noticed by SLUB in debug mode:
      
      Complete reproducer under usermode linux (discovered on real
      machine):
      
          bdev=/dev/ubda
          btr_root=/btr
          /mkfs.btrfs $bdev
          mount $bdev $btr_root
          mkdir $btr_root/subvols/
          cd $btr_root/subvols/
          /btrfs su cr foo
          /btrfs su cr bar
          mount $bdev -osubvol=subvols/foo $btr_root/subvols/bar
          umount $btr_root/subvols/bar
      
      which gives
      
      device fsid 4d55aa28-45b1-474b-b4ec-da912322195e devid 1 transid 7 /dev/ubda
      =============================================================================
      BUG kmalloc-2048: Object already free
      -----------------------------------------------------------------------------
      
      INFO: Allocated in btrfs_mount+0x389/0x7f0 age=0 cpu=0 pid=277
      INFO: Freed in btrfs_mount+0x51c/0x7f0 age=0 cpu=0 pid=277
      INFO: Slab 0x0000000062886200 objects=15 used=9 fp=0x0000000070b4d2d0 flags=0x4081
      INFO: Object 0x0000000070b4d2d0 @offset=21200 fp=0x0000000070b4a968
      ...
      Call Trace:
      70b31948:  [<6008c522>] print_trailer+0xe2/0x130
      70b31978:  [<6008c5aa>] object_err+0x3a/0x50
      70b319a8:  [<6008e242>] free_debug_processing+0x142/0x2a0
      70b319e0:  [<600ebf6f>] btrfs_mount+0x55f/0x7f0
      70b319f8:  [<6008e5c1>] __slab_free+0x221/0x2d0
      Signed-off-by: default avatarSergei Trofimovich <slyfox@gentoo.org>
      Cc: Arne Jansen <sensille@gmx.net>
      Cc: Chris Mason <chris.mason@oracle.com>
      Cc: David Sterba <dsterba@suse.cz>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      45ea6095
  5. 06 Nov, 2011 21 commits
    • Chris Mason's avatar
      Btrfs: check for a null fs root when writing to the backup root log · 7c7e82a7
      Chris Mason authored
      During log replay, can commit the transaction before the fs_root
      pointers are setup, so we have to make sure they are not null before
      trying to use them.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      7c7e82a7
    • Chris Mason's avatar
      Btrfs: fix race during transaction joins · d43317dc
      Chris Mason authored
      While we're allocating ram for a new transaction, we drop our spinlock.
      When we get the lock back, we do check to see if a transaction started
      while we slept, but we don't check to make sure it isn't blocked
      because a commit has already started.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      d43317dc
    • Ilya Dryomov's avatar
      Btrfs: fix a potential btrfs_bio leak on scrub fixups · 56d2a48f
      Ilya Dryomov authored
      In case we were able to map less than we wanted (length < PAGE_SIZE
      clause is true) btrfs_bio is still allocated and we have to free it.
      Signed-off-by: default avatarIlya Dryomov <idryomov@gmail.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      56d2a48f
    • Ilya Dryomov's avatar
    • Ilya Dryomov's avatar
      9510dc4c
    • Chris Mason's avatar
      Btrfs: stop the readahead threads on failed mount · 306c8b68
      Chris Mason authored
      If we don't stop them, they linger around corrupting
      memory by using pointers to freed things.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      306c8b68
    • Chris Mason's avatar
      Btrfs: fix extent_buffer leak in the metadata IO error handling · c674e04e
      Chris Mason authored
      The scrub readahead branch brought in a new error handling hook,
      but it was leaking extent_buffer references.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      c674e04e
    • Chris Mason's avatar
      Btrfs: fix the new inspection ioctls for 32 bit compat · 740c3d22
      Chris Mason authored
      The new ioctls to follow backrefs are not clean for 32/64 bit
      compat.  This reworks them for u64s everywhere.  They are brand new, so
      there are no problems with changing the interface now.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      740c3d22
    • Chris Mason's avatar
      Merge git://git.jan-o-sch.net/btrfs-unstable into integration · 806468f8
      Chris Mason authored
      Conflicts:
      	fs/btrfs/Makefile
      	fs/btrfs/extent_io.c
      	fs/btrfs/extent_io.h
      	fs/btrfs/scrub.c
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      806468f8
    • Chris Mason's avatar
      Merge branch 'for-chris' of git://github.com/sensille/linux into integration · 531f4b1a
      Chris Mason authored
      Conflicts:
      	fs/btrfs/ctree.h
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      531f4b1a
    • Josef Bacik's avatar
      Btrfs: fix delayed insertion reservation · c06a0e12
      Josef Bacik authored
      We all keep getting those stupid warnings from use_block_rsv when running
      stress.sh, and it's because the delayed insertion stuff is being stupid.  It's
      not the delayed insertion stuffs fault, it's all just stupid.  When marking an
      inode dirty for oh say updating the time on it, we just do a
      btrfs_join_transaction, which doesn't reserve any space.  This is stupid because
      we're going to have to have space reserve to make this change, but we do it
      because it's fast because chances are we're going to call it over and over again
      and it doesn't matter.  Well thanks to the delayed insertion stuff this is
      mostly the case, so we do actually need to make this reservation.  So if
      trans->bytes_reserved is 0 then try to do a normal reservation.  If not return
      ENOSPC which will make the btrfs_dirty_inode start a proper transaction which
      will let it do the whole ENOSPC dance and reserve enough space for the delayed
      insertion to steal the reservation from the transaction.
      
      The other stupid thing we do is not reserve space for the inode when writing to
      the thing.  Usually this is ok since we have to update the time so we'd have
      already done all this work before we get to the endio stuff, so it doesn't
      matter.  But this is stupid because we could write the data after the
      transaction commits where we changed the mtime of the inode so we have to cow
      all the way down to the inode anyway.  This used to be masked by the delalloc
      reservation stuff, but because we delay the update it doesn't get masked in this
      case.  So again the delayed insertion stuff bites us in the ass.  So if our
      trans->block_rsv is delalloc, just steal the reservation from the delalloc
      reserve.  Hopefully this won't bite us in the ass, but I've said that before.
      
      With this patch stress.sh no longer spits out those stupid warnings (famous last
      words).  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      c06a0e12
    • Chris Mason's avatar
      Btrfs: ClearPageError during writepage and clean_tree_block · bf0da8c1
      Chris Mason authored
      Failure testing was tripping up over stale PageError bits in
      metadata pages.  If we have an io error on a block, and later on
      end up reusing it, nobody ever clears PageError on those pages.
      
      During commit, we'll find PageError and think we had trouble writing
      the block, which will lead to aborts and other problems.
      
      This changes clean_tree_block and the btrfs writepage code to
      clear the PageError bit.  In both cases we're either completely
      done with the page or the page has good stuff and the error bit
      is no longer valid.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      bf0da8c1
    • Josef Bacik's avatar
      Btrfs: be smarter about committing the transaction in reserve_metadata_bytes · 663350ac
      Josef Bacik authored
      Because of the overcommit stuff I had to make it so that we committed the
      transaction all the time in reserve_metadata_bytes in case we had overcommitted
      because of delayed items.  This was because previously we had no way of knowing
      how much space was reserved for delayed items.  Now that we have the
      delayed_block_rsv we can check it to see if committing the transaction would get
      us anywhere.  This patch breaks out the committing logic into a helper function
      that will check to see if committing the transaction would free enough space for
      us to get anything done.  With this patch xfstests 83 goes from taking 445
      seconds to taking 28 seconds on my box.  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      663350ac
    • Josef Bacik's avatar
      Btrfs: make a delayed_block_rsv for the delayed item insertion · 6d668dda
      Josef Bacik authored
      I've been hitting warnings in use_block_rsv when running the delayed insertion
      stuff.  It's because we will readjust global block rsv based on what is in use,
      which means we could end up discarding reservations that are for the delayed
      insertion stuff.  So instead create a seperate block rsv for the delayed
      insertion stuff.  This will also make it easier to debug problems with the
      delayed insertion reservations since we will know that only the delayed
      insertion code touches this block_rsv.  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      6d668dda
    • Chris Mason's avatar
      Btrfs: add a log of past tree roots · af31f5e5
      Chris Mason authored
      This takes some of the free space in the btrfs super block
      to record information about most of the roots in the last four
      commits.
      
      It also adds a -o recovery to use the root history log when
      we're not able to read the tree of tree roots, the extent
      tree root, the device tree root or the csum root.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      af31f5e5
    • David Sterba's avatar
      btrfs: separate superblock items out of fs_info · 6c41761f
      David Sterba authored
      fs_info has now ~9kb, more than fits into one page. This will cause
      mount failure when memory is too fragmented. Top space consumers are
      super block structures super_copy and super_for_commit, ~2.8kb each.
      Allocate them dynamically. fs_info will be ~3.5kb. (measured on x86_64)
      
      Add a wrapper for freeing fs_info and all of it's dynamically allocated
      members.
      Signed-off-by: default avatarDavid Sterba <dsterba@suse.cz>
      6c41761f
    • Josef Bacik's avatar
      Btrfs: use the global reserve when truncating the free space cache inode · c8174313
      Josef Bacik authored
      We no longer use the orphan block rsv for holding the reservation for truncating
      the inode, so instead use the global block rsv and check to make sure it has
      enough space for us to truncate the space.  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      c8174313
    • Josef Bacik's avatar
      Btrfs: release metadata from global reserve if we have to fallback for unlink · 5a77d76c
      Josef Bacik authored
      I fixed a problem where we weren't reserving space for an orphan item when we
      had to fallback to using the global reserve for an unlink, but I introduced
      another problem.  I was migrating the bytes from the transaction reserve to the
      global reserve and then releasing from the global reserve in
      btrfs_end_transaction().  The problem with this is that a migrate will jack up
      the size for the destination, but leave the size alone for the source, with the
      idea that you can do a release normally on the source and it all washes out, and
      then you can do a release again on the destination and it works out right.  My
      way was skipping the release on the trans_block_rsv which still had the jacked
      up size from our original reservation.  So instead release manually from the
      global reserve if this transaction was using it, and then set the
      trans->block_rsv back to the trans_block_rsv so that btrfs_end_transaction
      cleans everything up properly.  With this patch xfstest 83 doesn't emit warnings
      about leaking space.  Thanks,
      Signed-off-by: default avatarJosef Bacik <josef@redhat.com>
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      5a77d76c
    • Chris Mason's avatar
      Btrfs: make sure to flush queued bios if write_cache_pages waits · 01d658f2
      Chris Mason authored
      write_cache_pages tries to build up a large bio to stuff down the pipe.
      But if it needs to wait for a page lock, it needs to make sure and send
      down any pending writes so we don't deadlock with anyone who has the
      page lock and is waiting for writeback of things inside the bio.
      
      Dave Sterba triggered this as a deadlock between the autodefrag code and
      the extent write_cache_pages
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      01d658f2
    • Chris Mason's avatar
      Btrfs: fix extent pinning bugs in the tree log · e688b725
      Chris Mason authored
      The tree log had two important bugs that could cause corruptions after a
      crash.  Sometimes we were allowing tree log blocks to be reused after
      the tree log was committed but before the transaction commit was done.
      
      This allowed a future metadata write to overwrite the tree log data.  It
      is fixed by adding a new variant of freeing reserved extents that always
      pins them.  Credit goes to Stefan Behrens and Arne Jansen for many many
      hours spent tracking this bug down.
      
      During tree log replay, we do a pass through the tree log and pin all
      the extents we find.  This makes sure the replay code won't go in and
      use any of those blocks for new allocations during replay.  The problem
      is the free space cache isn't honoring these pinned extents.  So the
      allocator can end up handing them out, leading to all kinds of problems
      during replay.
      
      The fix here is to force any free space cache to load while we pin the
      extents, and then to make sure we remove the pinned extents from the
      free space rbtree.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      Reported-by: default avatarStefan Behrens <sbehrens@giantdisaster.de>
      e688b725
    • Chris Mason's avatar
      Btrfs: make sure btrfs_remove_free_space doesn't leak EAGAIN · 1eae31e9
      Chris Mason authored
      btrfs_remove_free_space needs to make sure to set ret back to a
      valid return value after setting it to EAGAIN, otherwise we return
      it to the callers.
      Signed-off-by: default avatarChris Mason <chris.mason@oracle.com>
      1eae31e9