Commit 04a646df authored by Harish Chegondi's avatar Harish Chegondi Committed by Doug Ledford

IB/hfi1: Fix the bail out code in pin_vector_pages() function

In pin_vector_pages() function, if there is any error while pinning the
pages or while adding a pinned buffer to the cache, the bail out code
needs to unpin any pinned pages that are not in the cache and adjust the
n_locked counter that counts the total pages pinned. The current bail
out code doesn't seem to be doing it right in two cases:

1. Before pinning required pages for a buffer, the SDMA pinned buffer
cache is searched to see if the virtual address range that needs to be
pinned is already pinned. If there isn't a hit in the cache, a new node
is created for the buffer and is added to the cache after the buffer is
pinned. If adding the new node to the cache fails, the n_locked count is
decremented properly but the pinned pages are not freed. This commit
fixes this issue.

2. If there is a hit in the SDMA cache, but the cached buffer doesn't
have enough pages to cover the entire address range that needs to be
pinned, the node for the cached buffer is extracted from the cache,
remaining pages needed are pinned and added to the node. The node is
finally added back into the cache. If there is an error pinning the
extra pages, the bail out code frees all the pages in the node but the
n_locked count is not being decremented by the no of pages in the node
that are freed. This commit fixes this issue.

This commit fixes the above two issues by creating a new function that
frees the pages in a node and decrements the n_locked count by the
number of pages freed.
Reviewed-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: default avatarHarish Chegondi <harish.chegondi@intel.com>
Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
parent 4c6c9aa6
...@@ -1167,6 +1167,14 @@ static int pin_sdma_pages(struct user_sdma_request *req, ...@@ -1167,6 +1167,14 @@ static int pin_sdma_pages(struct user_sdma_request *req,
return pinned; return pinned;
} }
static void unpin_sdma_pages(struct sdma_mmu_node *node)
{
if (node->npages) {
unpin_vector_pages(node->pq->mm, node->pages, 0, node->npages);
atomic_sub(node->npages, &node->pq->n_locked);
}
}
static int pin_vector_pages(struct user_sdma_request *req, static int pin_vector_pages(struct user_sdma_request *req,
struct user_sdma_iovec *iovec) struct user_sdma_iovec *iovec)
{ {
...@@ -1218,14 +1226,12 @@ static int pin_vector_pages(struct user_sdma_request *req, ...@@ -1218,14 +1226,12 @@ static int pin_vector_pages(struct user_sdma_request *req,
ret = hfi1_mmu_rb_insert(req->pq->handler, &node->rb); ret = hfi1_mmu_rb_insert(req->pq->handler, &node->rb);
if (ret) { if (ret) {
atomic_sub(node->npages, &pq->n_locked);
iovec->node = NULL; iovec->node = NULL;
goto bail; goto bail;
} }
return 0; return 0;
bail: bail:
if (rb_node) unpin_sdma_pages(node);
unpin_vector_pages(pq->mm, node->pages, 0, node->npages);
kfree(node); kfree(node);
return ret; return ret;
} }
...@@ -1671,10 +1677,7 @@ static void sdma_rb_remove(void *arg, struct mmu_rb_node *mnode) ...@@ -1671,10 +1677,7 @@ static void sdma_rb_remove(void *arg, struct mmu_rb_node *mnode)
struct sdma_mmu_node *node = struct sdma_mmu_node *node =
container_of(mnode, struct sdma_mmu_node, rb); container_of(mnode, struct sdma_mmu_node, rb);
atomic_sub(node->npages, &node->pq->n_locked); unpin_sdma_pages(node);
unpin_vector_pages(node->pq->mm, node->pages, 0, node->npages);
kfree(node); kfree(node);
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment