- 13 Oct, 2017 31 commits
-
-
Javier González authored
Refactor the rqd allocation and free functions so that all I/O types can use these helper functions. Signed-off-by:
Javier González <javier@cnexlabs.com> Signed-off-by:
Matias Bjørling <m@bjorling.me> Signed-off-by:
Jens Axboe <axboe@kernel.dk>
-
Javier González authored
Each request type sent to the LightNVM subsystem requires different metadata. Until now, we have tailored this metadata based on write, read and erase commands. However, pblk uses different metadata for internal writes that do not hit the write buffer. Instead of abusing the metadata for reads, create a new request type - internal write to improve code readability. In the process, create internal values for each I/O type instead of abusing the READ/WRITE macros, as suggested by Christoph. Signed-off-by:
-
Javier González authored
Wait until we know the exact number of ppas to be sent to the device, before allocating the bio. Signed-off-by:
-
Javier González authored
On REQ_PREFLUSH, directly tag the I/O context flags to signal a flush in the write to cache path, instead of finding the correct entry context and imposing a memory barrier. This simplifies the code and might potentially prevent race conditions when adding functionality to the write path. Signed-off-by:
-
Javier González authored
Simplify put bio by doing it on bio end_io instead of manually putting it on the completion path. Signed-off-by:
-
Javier González authored
Simplify the part of the garbage collector where data is read from the line being recycled and moved into an internal queue before being copied to the memory buffer. This allows to get rid of a dedicated function, which introduces an unnecessary dependency on the code. Signed-off-by:
-
Javier González authored
When a line is selected for recycling by the garbage collector (GC), the line state changes and the invalid bitmap is frozen, preventing invalidations from happening. Throughout the GC, the L2P map is checked to verify that not data being recycled has been updated. The last check is done before the new map is being stored on the L2P table. Though this algorithm works, it requires a number of corner cases to be checked each time the L2P table is being updated. This complicates readability and is error prone in case that the recycling algorithm is modified. Instead, this patch makes the invalid bitmap accessible even when the line is being recycled. When recycled data is being remapped, it is enough to check the invalid bitmap for the line before updating the L2P table. Signed-off-by:
-
Javier González authored
Refactor lba sanity check on read path to avoid code duplication. Signed-off-by:
-
Javier González authored
Normalize the way we name ppa variables to improve code readability. Signed-off-by:
-
Javier González authored
Use a constant to set the maximum number of inflight GC requests allowed. Signed-off-by:
-
Javier González authored
As part of the mempool audit on pblk, remove unnecessary mempool allocation checks on mempools. Reported-by:
-
Javier González authored
pblk holds two sector bitmaps: one to keep track of the mapped sectors while the line is active and another one to keep track of the invalid sectors. The latter is kept during the whole live of the line, until it is recycled. Since we cannot guarantee forward progress for the mempool in this case, get rid of the mempool and simply allocate memory through kmalloc. Reported-by:
-
Javier González authored
Since read and erase paths offer different guarantees for inflight I/Os, separate the mempools to set the right min_nr for each on creation. Reported-by:
-
Javier González authored
In pblk, we have a mempool to allocate a generic structure that we pass along workqueues. This is heavily used in the GC path in order to have enough inflight reads and fully utilize the GC bandwidth. However, the current GC path copies data to the host memory and puts it back into the write buffer. This requires a vmalloc allocation for the data and a memory copy. Thus, guaranteeing the allocation by using a mempool for the structure in itself does not give us much. Until we implement support for vector copy to avoid moving data through the host, just allocate the workqueue structure using kmalloc. This allows us to have a much smaller mempool. Reported-by:
-
Javier González authored
pblk uses an internal page mempool for allocating pages on internal bios. The main two users of this memory pool are partial reads (reads with some sectors in cache and some on media) and padded writes, which need to add dummy pages to an existing bio already containing valid data (and with a large enough bioset allocated). In both cases, the maximum number of pages per bio is defined by the maximum number of physical sectors supported by the underlying device. This patch fixes a bad mempool allocation, where the min_nr of elements on the pool was fixed (to 16), which is lower than the maximum number of sectors supported by NVMe (as of the time for this patch). Instead, use the maximum number of allowed sectors reported by the device. Reported-by:
-
Javier González authored
On low LUN configurations, make sure not to send bios that are bigger than the buffer size. Fixes: a4bd217b ("lightnvm: physical block device (pblk) target") Signed-off-by:
-
Javier González authored
Fix stat counter to collect the right number of I/Os being synced on the completion path. Fixes: 0880a9aa ("lightnvm: pblk: delete redundant buffer pointer") Signed-off-by:
-
Javier González authored
When a REQ_FLUSH reaches pblk, the bio cannot be directly completed. Instead, data on the write buffer is flushed and the bio is completed on the completion pah. This might require some sectors to be padded in order to guarantee a successful write. This patch fixes a memory leak on the padded pages. A consequence of this bad free was that internal bios not containing data (only a flush) were not being completed. Fixes: a4bd217b ("lightnvm: physical block device (pblk) target") Signed-off-by:
-
Javier González authored
The data buffer for the GC path allocates virtual memory through vmalloc. When this change was introduced, a flag signaling kmalloc'ed memory was wrongly introduced. Use the right flag when creating a bio from this buffer. Fixes: de54e703 ("lightnvm: pblk: use vmalloc for GC data buffer") Signed-off-by:
-
Javier González authored
Initialize the stat counter for garbage collected reads. Fixes: a4bd217b ("lightnvm: physical block device (pblk) target") Signed-off-by:
-
Rakesh Pandit authored
This is a trivial change which reuses pblk_gc_should_kick instead of repeating it again in pblk_rl_free_lines_inc. Signed-off-by:
Rakesh Pandit <rakesh@tuxera.com> Made it apply to the common case. Reviewed-by:
-
Rakesh Pandit authored
Correct it by converting little endian to cpu endian and also define a macro for line version so that maintenance is easy. Signed-off-by:
-
Rakesh Pandit authored
The two pr_err messages are useless as they don't differentiate error code. Signed-off-by:
-
Rakesh Pandit authored
This usually happens if we are developing with qemu and ll2pmode has default value. Improve description. Signed-off-by:
-
Rakesh Pandit authored
It seems pblk_dealloc_page would race against pblk_alloc_pages for line bitmap for sector allocation.The chances are very low but might as well protect the bitmap properly. Signed-off-by:
-
Rakesh Pandit authored
Because NVM needs BLK_DEV_NVME, select it automatically if we mark NVM in config file before building kernel. Also append PCI to depends as select doesn't automatically add dependencies. Signed-off-by:
-
Rakesh Pandit authored
Use appropriate memory free calls based on allocation type used and also fix number of times free is called if kmalloc fails. Signed-off-by:
-
Rakesh Pandit authored
Remove repeated calculation for number of channels while creating a target device. Signed-off-by:
-
Rakesh Pandit authored
nvm_tgt_types list was protected by wrong lock for NVM_INFO ioctl call and can race with addition or removal of target types. Also unregistering target type was not protected correctly. Fixes: 5cd90785 ("lightnvm: remove nested lock conflict with mm") Signed-off-by:
-
Rakesh Pandit authored
When a virtual block device is formatted and mounted after creating with "nvme lnvm create... -t pblk", a removal from "nvm lnvm remove" would result in this: 446416.309757] bdi-block not registered [446416.309773] ------------[ cut here ]------------ [446416.309780] WARNING: CPU: 3 PID: 4319 at fs/fs-writeback.c:2159 __mark_inode_dirty+0x268/0x340 Ideally removal should return -EBUSY as block device is mounted after formatting. This patch tries to address this checking if whole device or any partition of it already mounted or not before removal. Whole device is checked using "bd_super" member of block device. This member is always set once block device has been mounted using a filesystem. Another member "bd_part_count" takes care of checking any if any partitions are under use. "bd_part_count" is only updated under locks when partitions are opened or closed (first open and last release). This at least does take care sending -EBUSY if removal is being attempted while whole block device or any partition is mounted. Signed-off-by:
-
Rakesh Pandit authored
If target type module e.g. pblk here is unloaded (rmmod) while module is in use (after creating target) system crashes. We fix this by using module API refcnt. Signed-off-by:
-
- 12 Oct, 2017 2 commits
-
-
Christos Gkekas authored
Remove variables that are set but never used. Signed-off-by:
Christos Gkekas <chris.gekas@gmail.com> Signed-off-by:
-
Rakesh Pandit authored
Replaced by pr_err usage in commit ef510424 ("block, dax: move "select DAX" from BLOCK to FS_DAX") Signed-off-by:
Ross Zwisler <ross.zwisler@linux.intel.com> Signed-off-by:
-
- 10 Oct, 2017 5 commits
-
-
Shaohua Li authored
Legacy queue sets request's request_list, mq doesn't. This makes mq does the same thing, so we can find cgroup of a request. Note, we really only use blkg field of request_list, it's pointless to allocate mempool for request_list in mq case. Signed-off-by:
Shaohua Li <shli@fb.com> Signed-off-by:
-
Shaohua Li authored
Fix two issues: - the per-cpu stat flush is unnecessary, nobody uses per-cpu stat except sum it to global stat. We can do the calculation there. The flush just wastes cpu time. - some fields are signed int/s64. I don't see the point. Reviewed-by:
Omar Sandoval <osandov@fb.com> Signed-off-by:
-
Jiufei Xue authored
A null pointer dereference can occur when blkcg is removed manually with writeback IOs inflight. This is caused by the following case: Writeback kworker submit the bio and set bio->bi_cg_private to tg in blk_throtl_assoc_bio. Then we remove the block cgroup manually, the blkg and tg would be freed if there is no request inflight. When the submitted bio come back, blk_throtl_bio_endio() fetch the tg which was already freed. Fix this by increasing the refcount of blkg in funcion blk_throtl_assoc_bio() so that the blkg will not be freed until the bio_endio called. Reviewed-by:
Jiufei Xue <jiufei.xjf@alibaba-inc.com> Signed-off-by:
-
weiping zhang authored
check pol->cpd_free_fn() instead of pol->cpd_alloc_fn() when free cpd. Reviewed-by:
Johannes Thumshirn <jthumshirn@suse.de> Signed-off-by:
weiping zhang <zhangweiping@didichuxing.com> Signed-off-by:
-
Rakesh Pandit authored
Since commit 925a6efb ("Btrfs: stop using try_to_writeback_inodes_sb_nr to flush delalloc") this function hasn't been used outside so stop exporting it. In addition we merge it into try_to_writeback_inodes_sb() which is the only caller. Also change return type of try_to_writeback_inodes_sb to void as the only user ext4 doesn't care. Reviewed-by:
Jan Kara <jack@suse.cz> Signed-off-by:
-
- 09 Oct, 2017 2 commits
-
-
Paolo Valente authored
The commit "block, bfq: decrease burst size when queues in burst exit" introduced the decrement of burst_size on the removal of a bfq_queue from the burst list. Unfortunately, this decrement can happen to be performed even when burst size is already equal to 0, because of unbalanced decrements. A description follows of the cause of these unbalanced decrements, namely a wrong assumption, and of the way how this wrong assumption leads to unbalanced decrements. The wrong assumption is that a bfq_queue can exit only if the process associated with the bfq_queue has exited. This is false, because a bfq_queue, say Q, may exit also as a consequence of a merge with another bfq_queue. In this case, Q exits because the I/O of its associated process has been redirected to another bfq_queue. The decrement unbalance occurs because Q may then be re-created after a split, and added back to the current burst list, *without* incrementing burst_size. burst_size is not incremented because Q is not a new bfq_queue added to the burst list, but a bfq_queue only temporarily removed from the list, and, before the commit "bfq-sq, bfq-mq: decrease burst size when queues in burst exit", burst_size was not decremented when Q was removed. This commit addresses this issue by just checking whether the exiting bfq_queue is a merged bfq_queue, and, in that case, not decrementing burst_size. Unfortunately, this still leaves room for unbalanced decrements, in the following rarer case: on a split, the bfq_queue happens to be inserted into a different burst list than that it was removed from when merged. If this happens, the number of elements in the new burst list becomes higher than burst_size (by one). When the bfq_queue then exits, it is of course not in a merged state any longer, thus burst_size is decremented, which results in an unbalanced decrement. To handle this sporadic, unlucky case in a simple way, this commit also checks that burst_size is larger than 0 before decrementing it. Finally, this commit removes an useless, extra check: the check that the bfq_queue is sync, performed before checking whether the bfq_queue is in the burst list. This extra check is redundant, because only sync bfq_queues can be inserted into the burst list. Fixes: 7cb04004 ("block, bfq: decrease burst size when queues in burst exit") Reported-by:
Philip Müller <philm@manjaro.org> Signed-off-by:
Paolo Valente <paolo.valente@linaro.org> Signed-off-by:
Angelo Ruocco <angeloruocco90@gmail.com> Tested-by:
Oleksandr Natalenko <oleksandr@natalenko.name> Tested-by:
Lee Tibbert <lee.tibbert@gmail.com> Signed-off-by:
-
Luca Miccio authored
Similarly to CFQ, BFQ has its write-throttling heuristics, and it is better not to combine them with further write-throttling heuristics of a different nature. So this commit disables write-back throttling for a device if BFQ is used as I/O scheduler for that device. Signed-off-by:
Luca Miccio <lucmiccio@gmail.com> Signed-off-by:
-
