- 19 May, 2020 2 commits
-
-
Darrick J. Wong authored
While QAing the new xfs_repair quotacheck code, I uncovered a quota corruption bug resulting from a bad interaction between dquot buffer initialization and quotacheck. The bug can be reproduced with the following sequence: # mkfs.xfs -f /dev/sdf # mount /dev/sdf /opt -o usrquota # su nobody -s /bin/bash -c 'touch /opt/barf' # sync # xfs_quota -x -c 'report -ahi' /opt User quota on /opt (/dev/sdf) Inodes User ID Used Soft Hard Warn/Grace ---------- --------------------------------- root 3 0 0 00 [------] nobody 1 0 0 00 [------] # xfs_io -x -c 'shutdown' /opt # umount /opt # mount /dev/sdf /opt -o usrquota # touch /opt/man2 # xfs_quota -x -c 'report -ahi' /opt User quota on /opt (/dev/sdf) Inodes User ID Used Soft Hard Warn/Grace ---------- --------------------------------- root 1 0 0 00 [------] nobody 1 0 0 00 [------] # umount /opt Notice how the initial quotacheck set the root dquot icount to 3 (rootino, rbmino, rsumino), but after shutdown -> remount -> recovery, xfs_quota reports that the root dquot has only 1 icount. We haven't deleted anything from the filesystem, which means that quota is now under-counting. This behavior is not limited to icount or the root dquot, but this is the shortest reproducer. I traced the cause of this discrepancy to the way that we handle ondisk dquot updates during quotacheck vs. regular fs activity. Normally, when we allocate a disk block for a dquot, we log the buffer as a regular (dquot) buffer. Subsequent updates to the dquots backed by that block are done via separate dquot log item updates, which means that they depend on the logged buffer update being written to disk before the dquot items. Because individual dquots have their own LSN fields, that initial dquot buffer must always be recovered. However, the story changes for quotacheck, which can cause dquot block allocations but persists the final dquot counter values via a delwri list. Because recovery doesn't gate dquot buffer replay on an LSN, this means that the initial dquot buffer can be replayed over the (newer) contents that were delwritten at the end of quotacheck. In effect, this re-initializes the dquot counters after they've been updated. If the log does not contain any other dquot items to recover, the obsolete dquot contents will not be corrected by log recovery. Because quotacheck uses a transaction to log the setting of the CHKD flags in the superblock, we skip quotacheck during the second mount call, which allows the incorrect icount to remain. Fix this by changing the ondisk dquot initialization function to use ordered buffers to write out fresh dquot blocks if it detects that we're running quotacheck. If the system goes down before quotacheck can complete, the CHKD flags will not be set in the superblock and the next mount will run quotacheck again, which can fix uninitialized dquot buffers. This requires amending the defer code to maintaine ordered buffer state across defer rolls for the sake of the dquot allocation code. For regular operations we preserve the current behavior since the dquot items require properly initialized ondisk dquot records. Signed-off-by:Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by:
Brian Foster <bfoster@redhat.com> Reviewed-by:
Christoph Hellwig <hch@lst.de>
-
Brian Foster authored
The attr fork can transition from shortform to leaf format while empty if the first xattr doesn't fit in shortform. While this empty leaf block state is intended to be transient, it is technically not due to the transactional implementation of the xattr set operation. We historically have a couple of bandaids to work around this problem. The first is to hold the buffer after the format conversion to prevent premature writeback of the empty leaf buffer and the second is to bypass the xattr count check in the verifier during recovery. The latter assumes that the xattr set is also in the log and will be recovered into the buffer soon after the empty leaf buffer is reconstructed. This is not guaranteed, however. If the filesystem crashes after the format conversion but before the xattr set that induced it, only the format conversion may exist in the log. When recovered, this creates a latent corrupted state on the inode as any subsequent attempts to read the buffer fail due to verifier failure. This includes further attempts to set xattrs on the inode or attempts to destroy the attr fork, which prevents the inode from ever being removed from the unlinked list. To avoid this condition, accept that an empty attr leaf block is a valid state and remove the count check from the verifier. This means that on rare occasions an attr fork might exist in an unexpected state, but is otherwise consistent and functional. Note that we retain the logic to avoid racing with metadata writeback to reduce the window where this can occur. Signed-off-by:
-
- 13 May, 2020 3 commits
-
-
Nishad Kamdar authored
This patch corrects the SPDX License Identifier style in header files related to XFS File System support. For C header files Documentation/process/license-rules.rst mandates C-like comments. (opposed to C source files where C++ style should be used). Changes made by using a script provided by Joe Perches here: https://lkml.org/lkml/2019/2/7/46. Suggested-by:
Joe Perches <joe@perches.com> Signed-off-by:
Nishad Kamdar <nishadkamdar@gmail.com> Reviewed-by:
-
Gustavo A. R. Silva authored
The current codebase makes use of the zero-length array language extension to the C90 standard, but the preferred mechanism to declare variable-length types such as these ones is a flexible array member[1][2], introduced in C99: struct foo { int stuff; struct boo array[]; }; By making use of the mechanism above, we will get a compiler warning in case the flexible array does not occur last in the structure, which will help us prevent some kind of undefined behavior bugs from being inadvertently introduced[3] to the codebase from now on. Also, notice that, dynamic memory allocations won't be affected by this change: "Flexible array members have incomplete type, and so the sizeof operator may not be applied. As a quirk of the original implementation of zero-length arrays, sizeof evaluates to zero."[1] sizeof(flexible-array-member) triggers a warning because flexible array members have incomplete type[1]. There are some instances of code in which the sizeof operator is being incorrectly/erroneously applied to zero-length arrays and the result is zero. Such instances may be hiding some bugs. So, this work (flexible-array member conversions) will also help to get completely rid of those sorts of issues. This issue was found with the help of Coccinelle. [1] https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html [2] https://github.com/KSPP/linux/issues/21 [3] commit 76497732 ("cxgb3/l2t: Fix undefined behaviour") Signed-off-by:Gustavo A. R. Silva <gustavoars@kernel.org> Reviewed-by:
-
Zheng Bin authored
Construct an img like this: dd if=/dev/zero of=xfs.img bs=1M count=20 mkfs.xfs -d agcount=1 xfs.img xfs_db -x xfs.img sb 0 write fdblocks 0 agf 0 write freeblks 0 write longest 0 quit mount it, df -h /mnt(xfs mount point), will show this: Filesystem Size Used Avail Use% Mounted on /dev/loop0 17M -64Z -32K 100% /mnt Reported-by:
Hulk Robot <hulkci@huawei.com> Signed-off-by:
Zheng Bin <zhengbin13@huawei.com> Reviewed-by:
-
- 08 May, 2020 27 commits
-
-
Chen Zhou authored
Remove duplicate headers which are included twice. Signed-off-by:
Chen Zhou <chenzhou10@huawei.com> Reviewed-by:
-
Brian Foster authored
The random buffer write failure errortag patch introduced a local mount pointer variable for the test macro, but the macro is compiled out on !DEBUG kernels. This results in an unused variable warning. Access the mount structure through the buffer pointer and remove the local mount pointer to address the warning. Fixes: 7376d745 ("xfs: random buffer write failure errortag") Signed-off-by:
-
Darrick J. Wong authored
Remove unnecessary includes from the log recovery code. Suggested-by:
Christoph Hellwig <hch@infradead.org> Signed-off-by:
Chandan Babu R <chandanrlinux@gmail.com>
-
Darrick J. Wong authored
Move the helpers that handle incore buffer cancellation records to xfs_buf_item_recover.c since they're not directly related to the main log recovery machinery. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
The only purpose of XFS_LI_RECOVERED is to prevent log recovery from trying to replay recovered intents more than once. Therefore, we can move the bit setting up to the ->iop_recover caller. Signed-off-by:
-
Darrick J. Wong authored
Now that we've made the recovered item tests all the same, we can hoist the test and the ail locking code to the ->iop_recover caller and call the recovery function directly. Signed-off-by:
-
Darrick J. Wong authored
Rename XFS_{EFI,BUI,RUI,CUI}_RECOVERED to XFS_LI_RECOVERED so that we track recovery status in the log item, then get rid of the now unused flags fields in each of those log item types. Signed-off-by: -
Darrick J. Wong authored
During recovery, every intent that we recover from the log has to be added to the AIL. Replace the open-coded addition with a helper. Signed-off-by:
-
Darrick J. Wong authored
Replace the open-coded AIL item walking with a proper helper when we're trying to release an intent item that has been finished. We add a new ->iop_match method to decide if an intent item matches a supplied ID. Signed-off-by:
-
Darrick J. Wong authored
Now that we've finished converting all types of log intent items to provide an ->iop_recover function, we can convert the "is this an intent item?" predicate to look for a non-null iop_recover pointer. Move the predicate closer to the functions that use it. Signed-off-by:
-
Darrick J. Wong authored
Move the code that processes the log items created from the recovered log items into the per-item source code files and use dispatch functions to call them. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the code that processes the log items created from the recovered log items into the per-item source code files and use dispatch functions to call them. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the code that processes the log items created from the recovered log items into the per-item source code files and use dispatch functions to call them. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the code that processes the log items created from the recovered log items into the per-item source code files and use dispatch functions to call them. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Quotaoff doesn't actually do anything, so take advantage of the commit_pass2 pointer being optional and get rid of the switch statement clause. Signed-off-by:
-
Darrick J. Wong authored
Move the bmap update intent and intent-done pass2 commit code into the per-item source code files and use dispatch functions to call them. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the refcount update intent and intent-done pass2 commit code into the per-item source code files and use dispatch functions to call them. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the rmap update intent and intent-done pass2 commit code into the per-item source code files and use dispatch functions to call them. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the extent free intent and intent-done pass2 commit code into the per-item source code files and use dispatch functions to call them. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the log icreate item pass2 commit code into the per-item source code files and use the dispatch function to call it. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the log dquot item pass2 commit code into the per-item source code files and use the dispatch function to call it. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the log inode item pass2 commit code into the per-item source code files and use the dispatch function to call it. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the log buffer item pass2 commit code into the per-item source code files and use the dispatch function to call it. We do these one at a time because there's a lot of code to move. No functional changes. Signed-off-by:
-
Darrick J. Wong authored
Move the pass1 commit code into the per-item source code files and use the dispatch function to call them. Signed-off-by:
-
Darrick J. Wong authored
Move the pass2 readhead code into the per-item source code files and use the dispatch function to call them. Signed-off-by:
-
Darrick J. Wong authored
Create a generic dispatch structure to delegate recovery of different log item types into various code modules. This will enable us to move code specific to a particular log item type out of xfs_log_recover.c and into the log item source. The first operation we virtualize is the log item sorting. Signed-off-by:
-
Darrick J. Wong authored
Remove the old typedefs. Signed-off-by:
-
- 07 May, 2020 8 commits
-
-
Brian Foster authored
iget_flags is unused in xfs_imap_to_bp(). Remove the parameter and fix up the callers. Signed-off-by:
Allison Collins <allison.henderson@oracle.com> Signed-off-by:
-
Brian Foster authored
Both types control shutdown messaging and neither is used in the current codebase. Signed-off-by:
Dave Chinner <dchinner@redhat.com> Reviewed-by:
-
Brian Foster authored
Introduce an error tag to randomly fail async buffer writes. This is primarily to facilitate testing of the XFS error configuration mechanism. Signed-off-by:
-
Brian Foster authored
The stale parameter was used to control the now unused shutdown parameter of xfs_trans_ail_remove(). Signed-off-by:
-
Brian Foster authored
Now that the functions and callers of xfs_trans_ail_[remove|delete]() have been fixed up appropriately, the only difference between the two is the shutdown behavior. There are only a few callers of the _remove() variant, so make the shutdown conditional on the parameter and combine the two functions. Suggested-by:
Dave Chinner <david@fromorbit.com> Signed-off-by:
-
Brian Foster authored
The shutdown parameter of xfs_trans_ail_remove() is no longer used. The remaining callers use it for items that legitimately might not be in the AIL or from contexts where AIL state has already been checked. Remove the unnecessary parameter and fix up the callers. Signed-off-by:
-
Brian Foster authored
Various intent log items call xfs_trans_ail_remove() with a log I/O error shutdown type, but this helper historically checks whether an item is in the AIL before calling xfs_trans_ail_delete(). This means the shutdown check is essentially a no-op for users of xfs_trans_ail_remove(). It is possible that some items might not be AIL resident when the AIL remove attempt occurs, but this should be isolated to cases where the filesystem has already shutdown. For example, this includes abort of the transaction committing the intent and I/O error of the iclog buffer committing the intent to the log. Therefore, update these callsites to use xfs_trans_ail_delete() to provide AIL state validation for the common path of items being released and removed when associated done items commit to the physical log. Signed-off-by:
-
Brian Foster authored
Several callers acquire the lock just prior to the call. Callers that require ->ail_lock for other purposes already check IN_AIL state and thus don't require the additional shutdown check in the helper. Push the lock down into xfs_trans_ail_delete(), open code the instances that still acquire it, and remove the unnecessary ailp parameter. Signed-off-by:
-
