- 04 May, 2021 1 commit
-
-
Darrick J. Wong authored
While running generic/050 with an external log, I observed this warning in dmesg: Trying to write to read-only block-device sda4 (partno 4) WARNING: CPU: 2 PID: 215677 at block/blk-core.c:704 submit_bio_checks+0x256/0x510 Call Trace: submit_bio_noacct+0x2c/0x430 _xfs_buf_ioapply+0x283/0x3c0 [xfs] __xfs_buf_submit+0x6a/0x210 [xfs] xfs_buf_delwri_submit_buffers+0xf8/0x270 [xfs] xfsaild+0x2db/0xc50 [xfs] kthread+0x14b/0x170 I think this happened because we tried to cover the log after a readonly mount, and the AIL tried to write the primary superblock to the data device. The test marks the data device readonly, but it doesn't do the same to the external log device. Therefore, XFS thinks that the log is writable, even though AIL writes whine to dmesg because the data device is read only. Fix this by amending xfs_log_writable to prevent writes when the AIL can't possible write anything into the filesystem. Note: As for the external log or the rt devices being readonly-- xfs_blkdev_get will complain about that if we aren't doing a norecovery mount. Signed-off-by:
Darrick J. Wong <djwong@kernel.org> Reviewed-by:
Brian Foster <bfoster@redhat.com>
-
- 29 Apr, 2021 8 commits
-
-
Darrick J. Wong authored
The final parameter of filemap_write_and_wait_range is the end of the range to flush, not the length of the range to flush. Fixes: 46afb062 ("xfs: only flush the unshared range in xfs_reflink_unshare") Signed-off-by:
Chandan Babu R <chandanrlinux@gmail.com> Reviewed-by:
-
Brian Foster authored
The blocks used for allocation btrees (bnobt and countbt) are technically considered free space. This is because as free space is used, allocbt blocks are removed and naturally become available for traditional allocation. However, this means that a significant portion of free space may consist of in-use btree blocks if free space is severely fragmented. On large filesystems with large perag reservations, this can lead to a rare but nasty condition where a significant amount of physical free space is available, but the majority of actual usable blocks consist of in-use allocbt blocks. We have a record of a (~12TB, 32 AG) filesystem with multiple AGs in a state with ~2.5GB or so free blocks tracked across ~300 total allocbt blocks, but effectively at 100% full because the the free space is entirely consumed by refcountbt perag reservation. Such a large perag reservation is by design on large filesystems. The problem is that because the free space is so fragmented, this AG contributes the 300 or so allocbt blocks to the global counters as free space. If this pattern repeats across enough AGs, the filesystem lands in a state where global block reservation can outrun physical block availability. For example, a streaming buffered write on the affected filesystem continues to allow delayed allocation beyond the point where writeback starts to fail due to physical block allocation failures. The expected behavior is for the delalloc block reservation to fail gracefully with -ENOSPC before physical block allocation failure is a possibility. To address this problem, set aside in-use allocbt blocks at reservation time and thus ensure they cannot be reserved until truly available for physical allocation. This allows alloc btree metadata to continue to reside in free space, but dynamically adjusts reservation availability based on internal state. Note that the logic requires that the allocbt counter is fully populated at reservation time before it is fully effective. We currently rely on the mount time AGF scan in the perag reservation initialization code for this dependency on filesystems where it's most important (i.e. with active perag reservations). Signed-off-by:
Allison Henderson <allison.henderson@oracle.com> Reviewed-by:
-
Brian Foster authored
Introduce an in-core counter to track the sum of all allocbt blocks used by the filesystem. This value is currently tracked per-ag via the ->agf_btreeblks field in the AGF, which also happens to include rmapbt blocks. A global, in-core count of allocbt blocks is required to identify the subset of global ->m_fdblocks that consists of unavailable blocks currently used for allocation btrees. To support this calculation at block reservation time, construct a similar global counter for allocbt blocks, populate it on first read of each AGF and update it as allocbt blocks are used and released. Signed-off-by:
-
Brian Foster authored
perag reservation is enabled at mount time on a per AG basis. The upcoming change to set aside allocbt blocks from block reservation requires a populated allocbt counter as soon as possible after mount to be fully effective against large perag reservations. Therefore as a preparation step, initialize the pagf on all mounts where at least one reservation is active. Note that this already occurs to some degree on most default format filesystems as reservation requirement calculations already depend on the AGF or AGI, depending on the reservation type. Signed-off-by:
-
Darrick J. Wong authored
Since agf_btreeblks didn't exist before the lazysbcount feature, the fs summary count scrubber needs to walk the free space btrees to determine the amount of space being used by those btrees. Signed-off-by:
Gao Xiang <hsiangkao@redhat.com>
-
Dave Chinner authored
Keep the mount superblock counters up to date for !lazysbcount filesystems so that when we log the superblock they do not need updating in any way because they are already correct. It's found by what Zorro reported: 1. mkfs.xfs -f -l lazy-count=0 -m crc=0 $dev 2. mount $dev $mnt 3. fsstress -d $mnt -p 100 -n 1000 (maybe need more or less io load) 4. umount $mnt 5. xfs_repair -n $dev and I've seen no problem with this patch. Signed-off-by:
Dave Chinner <dchinner@redhat.com> Reported-by:
Zorro Lang <zlang@redhat.com> Reviewed-by:
-
Darrick J. Wong authored
The AGF free space btree block counter wasn't added until the lazysbcount feature was added to XFS midway through the life of the V4 format, so ignore the field when checking. Online AGF repair requires rmapbt, so it doesn't need the feature check. Signed-off-by:
-
Darrick J. Wong authored
In commit f8f2835a we changed the behavior of XFS to use EFIs to remove blocks from an overfilled AGFL because there were complaints about transaction overruns that stemmed from trying to free multiple blocks in a single transaction. Unfortunately, that commit missed a subtlety in the debug-mode transaction accounting when a realtime volume is attached. If a realtime file undergoes a data fork mapping change such that realtime extents are allocated (or freed) in the same transaction that a data device block is also allocated (or freed), we can trip a debugging assertion. This can happen (for example) if a realtime extent is allocated and it is necessary to reshape the bmbt to hold the new mapping. When we go to allocate a bmbt block from an AG, the first thing the data device block allocator does is ensure that the freelist is the proper length. If the freelist is too long, it will trim the freelist to the proper length. In debug mode, trimming the freelist calls xfs_trans_agflist_delta() to record the decrement in the AG free list count. Prior to f8f28 we would put the free block back in the free space btrees in the same transaction, which calls xfs_trans_agblocks_delta() to record the increment in the AG free block count. Since AGFL blocks are included in the global free block count (fdblocks), there is no corresponding fdblocks update, so the AGFL free satisfies the following condition in xfs_trans_apply_sb_deltas: /* * Check that superblock mods match the mods made to AGF counters. */ ASSERT((tp->t_fdblocks_delta + tp->t_res_fdblocks_delta) == (tp->t_ag_freeblks_delta + tp->t_ag_flist_delta + tp->t_ag_btree_delta)); The comparison here used to be: (X + 0) == ((X+1) + -1 + 0), where X is the number blocks that were allocated. After commit f8f28 we defer the block freeing to the next chained transaction, which means that the calls to xfs_trans_agflist_delta and xfs_trans_agblocks_delta occur in separate transactions. The (first) transaction that shortens the free list trips on the comparison, which has now become: (X + 0) == ((X) + -1 + 0) because we haven't freed the AGFL block yet; we've only logged an intention to free it. When the second transaction (the deferred free) commits, it will evaluate the expression as: (0 + 0) == (1 + 0 + 0) and trip over that in turn. At this point, the astute reader may note that the two commits tagged by this patch have been in the kernel for a long time but haven't generated any bug reports. How is it that the author became aware of this bug? This originally surfaced as an intermittent failure when I was testing realtime rmap, but a different bug report by Zorro Lang reveals the same assertion occuring on !lazysbcount filesystems. The common factor to both reports (and why this problem wasn't previously reported) becomes apparent if we consider when xfs_trans_apply_sb_deltas is called by __xfs_trans_commit(): if (tp->t_flags & XFS_TRANS_SB_DIRTY) xfs_trans_apply_sb_deltas(tp); With a modern lazysbcount filesystem, transactions update only the percpu counters, so they don't need to set XFS_TRANS_SB_DIRTY, hence xfs_trans_apply_sb_deltas is rarely called. However, updates to the count of free realtime extents are not part of lazysbcount, so XFS_TRANS_SB_DIRTY will be set on transactions adding or removing data fork mappings to realtime files; similarly, XFS_TRANS_SB_DIRTY is always set on !lazysbcount filesystems. Dave mentioned in response to an earlier version of this patch: "IIUC, what you are saying is that this debug code is simply not exercised in normal testing and hasn't been for the past decade? And it still won't be exercised on anything other than realtime device testing? "...it was debugging code from 1994 that was largely turned into dead code when lazysbcounters were introduced in 2007. Hence I'm not sure it holds any value anymore." This debugging code isn't especially helpful - you can modify the flcount on one AG and the freeblks of another AG, and it won't trigger. Add the fact that nobody noticed for a decade, and let's just get rid of it (and start testing realtime :P). This bug was found by running generic/051 on either a V4 filesystem lacking lazysbcount; or a V5 filesystem with a realtime volume. Cc: bfoster@redhat.com, zlang@redhat.com Fixes: f8f2835a ("xfs: defer agfl block frees when dfops is available") Signed-off-by:
-
- 23 Apr, 2021 2 commits
-
-
Christoph Hellwig authored
Rename struct xfs_legacy_ictimestamp to struct xfs_log_legacy_timestamp as it is a type used for logging timestamps with no relationship to the in-core inode. Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
-
Christoph Hellwig authored
Rename xfs_ictimestamp_t to xfs_log_timestamp_t as it is a type used for logging timestamps with no relationship to the in-core inode. Signed-off-by:
-
- 16 Apr, 2021 1 commit
-
-
Darrick J. Wong authored
The function was renamed, so get rid of the declaration. Signed-off-by:
-
- 15 Apr, 2021 7 commits
-
-
Christoph Hellwig authored
The in-memory XFS_IFEXTENTS is now only used to check if an inode with extents still needs the extents to be read into memory before doing operations that need the extent map. Add a new xfs_need_iread_extents helper that returns true for btree format forks that do not have any entries in the in-memory extent btree, and use that instead of checking the XFS_IFEXTENTS flag. Signed-off-by:
-
Christoph Hellwig authored
Just check for an inline format fork instead of the using the equivalent in-memory XFS_IFINLINE flag. Signed-off-by:
-
Christoph Hellwig authored
Just check for a btree format fork instead of the using the equivalent in-memory XFS_IFBROOT flag. Signed-off-by:
-
Christoph Hellwig authored
Stop using the XFS_IFEXTENTS flag, and instead switch on the fork format in xfs_idestroy_fork to decide how to cleanup. Signed-off-by:
-
Christoph Hellwig authored
Directly return from the subfunctions and avoid the error variable. Also remove the not really needed dp local variable. Signed-off-by:
-
Christoph Hellwig authored
xfs_bmap_one_block is only called for the attribute fork. Move it to xfs_attr.c, drop the unused whichfork argument and code only executed for the data fork and rename the result to xfs_attr_is_leaf. Signed-off-by:
-
Christoph Hellwig authored
Move the XFS_IFEXTENTS check from the callers into xfs_iread_extents to simplify the code. Signed-off-by:
-
- 09 Apr, 2021 7 commits
-
-
Brian Foster authored
xfs_setfilesize() is the only remaining caller of the internal __xfs_setfilesize() helper. Fold them into a single function. Signed-off-by:
-
Brian Foster authored
XFS no longer attaches anthing to ioend->io_private. Remove the unnecessary ->io_private merging code. This removes the only remaining user of xfs_setfilesize_ioend() so remove that function as well. Signed-off-by:
-
Brian Foster authored
Open code xfs_ioend_needs_workqueue() into the only remaining caller. Signed-off-by:
-
Brian Foster authored
Per-inode ioend completion batching has a log reservation deadlock vector between preallocated append transactions and transactions that are acquired at completion time for other purposes (i.e., unwritten extent conversion or COW fork remaps). For example, if the ioend completion workqueue task executes on a batch of ioends that are sorted such that an append ioend sits at the tail, it's possible for the outstanding append transaction reservation to block allocation of transactions required to process preceding ioends in the list. Append ioend completion is historically the common path for on-disk inode size updates. While file extending writes may have completed sometime earlier, the on-disk inode size is only updated after successful writeback completion. These transactions are preallocated serially from writeback context to mitigate concurrency and associated log reservation pressure across completions processed by multi-threaded workqueue tasks. However, now that delalloc blocks unconditionally map to unwritten extents at physical block allocation time, size updates via append ioends are relatively rare. This means that inode size updates most commonly occur as part of the preexisting completion time transaction to convert unwritten extents. As a result, there is no longer a strong need to preallocate size update transactions. Remove the preallocation of inode size update transactions to avoid the ioend completion processing log reservation deadlock. Instead, continue to send all potential size extending ioends to workqueue context for completion and allocate the transaction from that context. This ensures that no outstanding log reservation is owned by the ioend completion worker task when it begins to process ioends. Signed-off-by:
-
Colin Ian King authored
A previous commit removed a call to xfs_attr3_leaf_read that assigned an error return code to variable error. We now have a few early error return paths to label 'out' that return error if error is set; however error now is uninitialized so potentially garbage is being returned. Fix this by setting error to zero to restore the original behaviour where error was zero at the label 'restart'. Addresses-Coverity: ("Uninitialized scalar variable") Fixes: 07120f1a ("xfs: Add xfs_has_attr and subroutines") Signed-off-by:Colin Ian King <colin.king@canonical.com> Reviewed-by:
-
Darrick J. Wong authored
Now that the scrub context stores a pointer to the file that was used to invoke the scrub call, the struct xfs_inode pointer that we passed to all the setup functions is no longer necessary. This is only ever used if the caller wants us to scrub the metadata of the open file. Signed-off-by:
-
Darrick J. Wong authored
While running a new fstest that races a readonly remount with scrub running in repair mode, I observed the kernel tripping over debugging assertions in the log quiesce code that were checking that the CIL was empty. When the sysadmin runs scrub in repair mode, the scrub code allocates real transactions (with reservations) to change things, but doesn't increment the superblock writers count to block a readonly remount attempt while it is running. We don't require the userspace caller to have a writable file descriptor to run repairs, so we have to call mnt_want_write_file to obtain freeze protection and increment the writers count. It's ok to remove the call to sb_start_write for the dry-run case because commit 8321ddb2 removed the behavior where scrub and fsfreeze fight over the buffer LRU. Signed-off-by:
-
- 07 Apr, 2021 14 commits
-
-
Darrick J. Wong authored
Fix the weird split of responsibilities between xfs_can_free_eofblocks and xfs_free_eofblocks by moving the chunk of code that looks for any actual post-EOF space mappings from the second function into the first. This clears the way for deferred inode inactivation to be able to decide if an inode needs inactivation work before committing the released inode to the inactivation code paths (vs. marking it for reclaim). Signed-off-by:
-
Darrick J. Wong authored
In xfs_inode_free_eofblocks, move the xfs_can_free_eofblocks call further down in the function to the point where we have taken the IOLOCK. This is preparation for the next patch, where we will need that lock (or equivalent) so that we can check if there are any post-eof blocks to clean out. Signed-off-by:
-
Dave Chinner authored
Default attr fork offset is based on inode size, so is a fixed geometry parameter of the inode. Move it to the xfs_ino_geometry structure and stop calculating it on every call to xfs_default_attroffset(). Signed-off-by:
-
Dave Chinner authored
Device inodes have a non-default data fork size of 8 bytes as checked/enforced by xfs_repair. xfs_default_attroffset() doesn't handle this, so lets do a minor refactor so it does. Fixes: e6a688c3 ("xfs: initialise attr fork on inode create") Signed-off-by:
-
Dave Chinner authored
Due to confusion on when the XFS_IFEXTENT needs to be set, the changes in e6a688c3 ("xfs: initialise attr fork on inode create") failed to set the flag when initialising the empty attribute fork at inode creation. Set this flag the same way xfs_bmap_add_attrfork() does after attry fork allocation. Fixes: e6a688c3 ("xfs: initialise attr fork on inode create") Signed-off-by:
-
Dave Chinner authored
The pitfalls of regression testing on a machine without realising that selinux was disabled. Only set the attr fork during inode allocation if the attr feature bits are already set on the superblock. Fixes: e6a688c3 ("xfs: initialise attr fork on inode create") Signed-off-by:
-
Chandan Babu R authored
xchk_btree_check_minrecs() checks if the contents of the immediate child of a bmbt root block can fit within the root block. This check could fail on inodes with an attr fork since xfs_bmap_add_attrfork_btree() used to demote the current root node of the data fork as the child of a newly allocated root node if it found that the size of "struct xfs_btree_block" along with the space required for records exceeded that of space available in the data fork. xfs_bmap_add_attrfork_btree() should have used "struct xfs_bmdr_block" instead of "struct xfs_btree_block" for the above mentioned space requirement calculation. This commit disables the check for unoptimized (in terms of disk space usage) data fork bmbt trees since there could be filesystems in use that already have such a layout. Suggested-by:
-
Chandan Babu R authored
The incore data fork of an inode stores the bmap btree root node as 'struct xfs_btree_block'. However, the ondisk version of the inode stores the bmap btree root node as a 'struct xfs_bmdr_block'. xfs_bmap_add_attrfork_btree() checks if the btree root node fits inside the data fork of the inode. However, it incorrectly uses 'struct xfs_btree_block' to compute the size of the bmap btree root node. Since size of 'struct xfs_btree_block' is larger than that of 'struct xfs_bmdr_block', xfs_bmap_add_attrfork_btree() could end up unnecessarily demoting the current root node as the child of newly allocated root node. This commit optimizes space usage by modifying xfs_bmap_add_attrfork_btree() to use 'struct xfs_bmdr_block' to check if the bmap btree root node fits inside the data fork of the inode. Reviewed-by:
-
Anthony Iliopoulos authored
Use of the flag has had no effect since kernel commit 288699fe ("xfs: drop dmapi hooks"), which removed all dmapi related code, so deprecate it. Signed-off-by:
Anthony Iliopoulos <ailiop@suse.com> Reviewed-by:
-
Christoph Hellwig authored
Merge _xfs_dic2xflags into its only caller. Signed-off-by:
-
Christoph Hellwig authored
Move the crtime field from struct xfs_icdinode into stuct xfs_inode and remove the now entirely unused struct xfs_icdinode. Signed-off-by:
-
Christoph Hellwig authored
In preparation of removing the historic icinode struct, move the flags2 field into the containing xfs_inode structure. Signed-off-by:
-
Christoph Hellwig authored
In preparation of removing the historic icinode struct, move the flags field into the containing xfs_inode structure. Signed-off-by:
-
Christoph Hellwig authored
In preparation of removing the historic icinode struct, move the forkoff field into the containing xfs_inode structure. Signed-off-by:
-
