- 01 Jul, 2019 39 commits
-
-
Johannes Thumshirn authored
Commit 9678c543 ("btrfs: Remove custom crc32c init code") removed the btrfs_crc32c() function, because it was a duplicate of the crc32c() library function we already have in the kernel. Resurrect it as a shim wrapper over crc32c() to make following transformations of the checksumming code in btrfs easier. Also provide a btrfs_crc32_final() to ease following transformations. Reviewed-by:
Nikolay Borisov <nborisov@suse.com> Signed-off-by:
Johannes Thumshirn <jthumshirn@suse.de> Reviewed-by:
David Sterba <dsterba@suse.com> Signed-off-by:
-
Johannes Thumshirn authored
btrfsic_test_for_metadata() directly calls the crc32c() library function for calculating the CRC32C checksum, but then uses btrfs_csum_final() to invert the result. To ease further refactoring and development around checksumming in BTRFS convert to calling btrfs_csum_data(), which is a wrapper around crc32c(). Reviewed-by:
-
Qu Wenruo authored
btrfs: Flush before reflinking any extent to prevent NOCOW write falling back to COW without data reservation [BUG] The following script can cause unexpected fsync failure: #!/bin/bash dev=/dev/test/test mnt=/mnt/btrfs mkfs.btrfs -f $dev -b 512M > /dev/null mount $dev $mnt -o nospace_cache # Prealloc one extent xfs_io -f -c "falloc 8k 64m" $mnt/file1 # Fill the remaining data space xfs_io -f -c "pwrite 0 -b 4k 512M" $mnt/padding sync # Write into the prealloc extent xfs_io -c "pwrite 1m 16m" $mnt/file1 # Reflink then fsync, fsync would fail due to ENOSPC xfs_io -c "reflink $mnt/file1 8k 0 4k" -c "fsync" $mnt/file1 umount $dev The fsync fails with ENOSPC, and the last page of the buffered write is lost. [CAUSE] This is caused by: - Btrfs' back reference only has extent level granularity So write into shared extent must be COWed even only part of the extent is shared. So for above script we have: - fallocate Create a preallocated extent where we can do NOCOW write. - fill all the remaining data and unallocated space - buffered write into preallocated space As we have not enough space available for data and the extent is not shared (yet) we fall into NOCOW mode. - reflink Now part of the large preallocated extent is shared, later write into that extent must be COWed. - fsync triggers writeback But now the extent is shared and therefore we must fallback into COW mode, which fails with ENOSPC since there's not enough space to allocate data extents. [WORKAROUND] The workaround is to ensure any buffered write in the related extents (not just the reflink source range) get flushed before reflink/dedupe, so that NOCOW writes succeed that happened before reflinking succeed. The workaround is expensive, we could do it better by only flushing NOCOW range, but that needs extra accounting for NOCOW range. For now, fix the possible data loss first. Reviewed-by:
Filipe Manana <fdmanana@suse.com> Signed-off-by:
Qu Wenruo <wqu@suse.com> Signed-off-by:
-
Nikolay Borisov authored
The first thing code does in check_can_nocow is trying to block concurrent snapshots. If this fails (due to snpashot already being in progress) the function returns ENOSPC which makes no sense. Instead return EAGAIN. Despite this return value not being propagated to callers it's good practice to return the closest in terms of semantics error code. No functional changes. Signed-off-by:
-
Nikolay Borisov authored
Signed-off-by:
-
Nikolay Borisov authored
In case no cached_state argument is passed to btrfs_lock_and_flush_ordered_range use one locally in the function. This optimises the case when an ordered extent is found since the unlock function will be able to unlock that state directly without searching for it again. Reviewed-by:
Josef Bacik <josef@toxicpanda.com> Signed-off-by:
-
Nikolay Borisov authored
There several functions which open code btrfs_lock_and_flush_ordered_range, just replace them with a call to the function. No functional changes. Reviewed-by:
-
Nikolay Borisov authored
There is a certain idiom used in multiple places in btrfs' codebase, dealing with flushing an ordered range. Factor this in a separate function that can be reused. Future patches will replace the existing code with that function. Reviewed-by:
-
Qu Wenruo authored
At the context of btrfs_run_delalloc_range(), we haven't started/joined a transaction, thus even something went wrong, we can't and won't abort transaction, thus no way to make the fs RO. Reviewed-by:
-
Qu Wenruo authored
Add trace event for update_bytes_pinned() and update_bytes_may_use() to detect underflow better. The output would be something like (only showing data part): ## Buffered write start, 16K total ## 2255.954 xfs_io/860 btrfs:update_bytes_may_use:(nil)U: type=DATA old=0 diff=4096 2257.169 sudo/860 btrfs:update_bytes_may_use:(nil)U: type=DATA old=4096 diff=4096 2257.346 sudo/860 btrfs:update_bytes_may_use:(nil)U: type=DATA old=8192 diff=4096 2257.542 sudo/860 btrfs:update_bytes_may_use:(nil)U: type=DATA old=12288 diff=4096 ## Delalloc start ## 3727.853 kworker/u8:3-e/700 btrfs:update_bytes_may_use:(nil)U: type=DATA old=16384 diff=-16384 ## Space cache update ## 3733.132 sudo/862 btrfs:update_bytes_may_use:(nil)U: type=DATA old=0 diff=65536 3733.169 sudo/862 btrfs:update_bytes_may_use:(nil)U: type=DATA old=65536 diff=-65536 3739.868 sudo/862 btrfs:update_bytes_may_use:(nil)U: type=DATA old=0 diff=65536 3739.891 sudo/862 btrfs:update_bytes_may_use:(nil)U: type=DATA old=65536 diff=-65536 These two trace events will allow bcc tool to probe btrfs_space_info changes and detect underflow with more details (e.g. backtrace for each update). Signed-off-by:
-
Qu Wenruo authored
Just add a safe net for btrfs_space_info member updating. Signed-off-by:
-
David Sterba authored
There are several places that call nr_data_stripes, but this value does not change. Signed-off-by: -
David Sterba authored
Signed-off-by: -
David Sterba authored
The helper lacks the btrfs_ prefix and the parameter is the raw blockgroup type, so none of the callers has to do the flags -> index conversion. Signed-off-by: -
David Sterba authored
Merge the repeated code before the if-else block. Signed-off-by: -
David Sterba authored
The raid_attr table is now 7 * 56 = 392 bytes long, consisting of just small numbers so we don't have to use ints. New size is 7 * 32 = 224, saving 3 cachelines. Signed-off-by: -
David Sterba authored
Factor the sequence of ifs to a helper, the 'data stripes' here means the number of stripes without redundancy and parity. Signed-off-by: -
David Sterba authored
The factor is the number of copies. Signed-off-by: -
David Sterba authored
Replace open coded list of the profiles by selecting them from the raid_attr table. The criteria are now more explicit, we need profiles that have more than 1 copy of the data or can reconstruct the data with a missing device. Signed-off-by: -
David Sterba authored
Iterate over the table and gather all allowed profiles for a given number of devices, instead of open coding. Signed-off-by: -
David Sterba authored
The number of tolerated failures is stored in the raid_attr table, use it. Signed-off-by: -
David Sterba authored
The dev_max constraints are defined in the raid_attr table, use it instead of open-coding it. Signed-off-by: -
David Sterba authored
fs_info::mapping_tree is the physical<->logical mapping tree and uses the same underlying structure as extents, but is embedded to another structure. There are no other members and this indirection is useless. No functional change. Signed-off-by: -
David Sterba authored
The minimum number of devices for RAID5 is 2, though this is only a bit expensive RAID1, and for RAID6 it's 3, which is a triple copy that works only 3 devices. mkfs.btrfs allows that and mounting such filesystem also works, so the conversion via balance filters is inconsistent with the others and we should not prevent it. Signed-off-by: -
David Sterba authored
The list of profiles in btrfs_chunk_max_errors lists DUP as a profile DUP able to tolerate 1 device missing. Though this profile is special with 2 copies, it still needs the device, unlike the others. Looking at the history of changes, thre's no clear reason why DUP is there, functions were refactored and blocks of code merged to one helper. d20983b4 Btrfs: fix writing data into the seed filesystem - factor code to a helper de11cc12 Btrfs: don't pre-allocate btrfs bio - unrelated change, DUP still in the list with max errors 1 a236aed1 Btrfs: Deal with failed writes in mirrored configurations - introduced the max errors, leaves DUP and RAID1 in the same group Reviewed-by:
-
Liu Bo authored
This code was first introduced in 5f39d397 ("Btrfs: Create extent_buffer interface for large blocksizes") and the function was named btrfs_unlink_trans. It later got renamed to __btrfs_unlink_inode and finally commit 16cdcec7 ("btrfs: implement delayed inode items operation") changed the way inodes are deleted and obviated the need for those two members. Signed-off-by:
Liu Bo <bo.liu@linux.alibaba.com> Reviewed-by:
-
Goldwyn Rodrigues authored
This is a leftover from 312c89fb ("btrfs: cleanup btrfs_mount() using btrfs_mount_root()"), the mode was used for opening devices that's not done here anymore. Reviewed-by:
Goldwyn Rodrigues <rgoldwyn@suse.com> Reviewed-by:
-
Su Yue authored
In function do_trimming(), block_group->lock should be unlocked first, as the locks should be released in the reverse order. This does not cause problems but should follow the best practices. Reviewed-by:
Su Yue <suy.fnst@cn.fujitsu.com> Reviewed-by:
-
Qu Wenruo authored
Under certain conditions, we could have strange file extent item in log tree like: item 18 key (69599 108 397312) itemoff 15208 itemsize 53 extent data disk bytenr 0 nr 0 extent data offset 0 nr 18446744073709547520 ram 18446744073709547520 The num_bytes + ram_bytes overflow 64 bit type. For num_bytes part, we can detect such overflow along with file offset (key->offset), as file_offset + num_bytes should never go beyond u64. For ram_bytes part, it's about the decompressed size of the extent, not directly related to the size. In theory it is OK to have a large value, and put extra limitation on RAM bytes may cause unexpected false alerts. So in tree-checker, we only check if the file offset and num bytes overflow. Signed-off-by:
-
Nikolay Borisov authored
This is already done in btrfs_init_dev_replace_tgtdev which is the first phase of device replace, called before doing scrub. During that time exclusive lock is held. Additionally btrfs_fs_device::commit_total_bytes is always set based on the size of the underlying block device which shouldn't change once set. This makes the 2nd assignment of the variable in the finishing phase redundant. Reviewed-by:
-
Nikolay Borisov authored
Part of device replace involves writing an item to the device root containing information about pending replace operations. Currently space for this item is not being explicitly reserved so this works thanks to presence of global reserve. While not fatal it's not a good practice. Let's be explicit about space requirement of device replace and reserve space when starting the transaction. Reviewed-by:
-
Nikolay Borisov authored
There are only 2 branches which goto leave label with need_unlock set to true. Essentially need_unlock is used as a substitute for directly calling up_write. Since the branches needing this are only 2 and their context is not that big it's more clear to just call up_write where required. No functional changes. Reviewed-by:
-
Nikolay Borisov authored
btrfs_init_dev_replace_tgtdev reads certain values from the source device (such as commit_total_bytes) which are updated during transaction commit. Currently this function is called before committing any pending transaction, leading to possibly reading outdated values. Fix this by moving the function below the transaction commit, at this point the EXCL_OP bit it set hence once transaction is complete the total size of the device cannot be changed (it's usually changed by resize/remove ops which are blocked). Fixes: 9e271ae2 ("Btrfs: kernel operation should come after user input has been verified") Signed-off-by:
-
Nikolay Borisov authored
This WARN_ON can never trigger because src_device cannot be null. btrfs_find_device_by_devspec always returns either an error or a valid pointer to the device. Just remove it. Reviewed-by:
-
Nikolay Borisov authored
There is no point in holding btrfs_fs_devices::device_list_mutex while initialising fields of the not-yet-published device. Instead, hold the mutex only when the newly initialised device is being published. I think holding device_list_mutex here is redundant altogether, because at this point BTRFS_FS_EXCL_OP is set which prevents device removal/addition/balance/resize to occur. Reviewed-by:
-
Nikolay Borisov authored
Using sync_blockdev makes it plain obvious what's happening. No functional changes. Reviewed-by:
-
David Sterba authored
btrfs_check_shared looks up parents of a given extent and uses ulists for that. These are allocated and freed repeatedly. Preallocation in the caller will avoid the overhead and also allow us to use the GFP_KERNEL as it is happens before the extent locks are taken. Reviewed-by:
-
David Sterba authored
Currently, there's only check for fast crc32c implementation on X86, based on the CPU flags. This is used to decide if checksumming should be offloaded to worker threads or can be calculated by the caller. As there are more architectures that implement a faster version of crc32c (ARM, SPARC, s390, MIPS, PowerPC), also there are specialized hw cards. The detection is based on driver name, all generic C implementations contain 'generic', while the specialized versions do not. Alternatively the priority could be used, but this is not currently provided by the crypto API. The flag is set per-filesystem at mount time and used for the offloading decisions. Signed-off-by: -
Qu Wenruo authored
Instead of using @sign to determine whether we're adding or subtracting. Even it only has 3 callers, it's still (and in fact already caused problem in the past) confusing to use. Refactor add_pinned_bytes() to add_pinned_bytes() and sub_pinned_bytes() to explicitly show what we're doing. Signed-off-by:
-
- 30 Jun, 2019 1 commit
-
-
Linus Torvalds authored
-
