Commit 325ad061 authored by Guy Shapiro's avatar Guy Shapiro Committed by Doug Ledford

IB/core: dma unmap optimizations

While unmapping an ODP writable page, the dirty bit of the page is set. In
order to do so, the head of the compound page is found.
Currently, the compound head is found even on non-writable pages, where it is
never used, leading to unnecessary cpu barrier that impacts performance.

This patch moves the search for the compound head to be done only when needed.
Signed-off-by: default avatarGuy Shapiro <guysh@mellanox.com>
Acked-by: default avatarShachar Raindel <raindel@mellanox.com>
Reviewed-by: default avatarSagi Grimberg <sagig@mellanox.com>
Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
parent c1d383b5
...@@ -637,7 +637,6 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt, ...@@ -637,7 +637,6 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt,
idx = (addr - ib_umem_start(umem)) / PAGE_SIZE; idx = (addr - ib_umem_start(umem)) / PAGE_SIZE;
if (umem->odp_data->page_list[idx]) { if (umem->odp_data->page_list[idx]) {
struct page *page = umem->odp_data->page_list[idx]; struct page *page = umem->odp_data->page_list[idx];
struct page *head_page = compound_head(page);
dma_addr_t dma = umem->odp_data->dma_list[idx]; dma_addr_t dma = umem->odp_data->dma_list[idx];
dma_addr_t dma_addr = dma & ODP_DMA_ADDR_MASK; dma_addr_t dma_addr = dma & ODP_DMA_ADDR_MASK;
...@@ -645,7 +644,8 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt, ...@@ -645,7 +644,8 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt,
ib_dma_unmap_page(dev, dma_addr, PAGE_SIZE, ib_dma_unmap_page(dev, dma_addr, PAGE_SIZE,
DMA_BIDIRECTIONAL); DMA_BIDIRECTIONAL);
if (dma & ODP_WRITE_ALLOWED_BIT) if (dma & ODP_WRITE_ALLOWED_BIT) {
struct page *head_page = compound_head(page);
/* /*
* set_page_dirty prefers being called with * set_page_dirty prefers being called with
* the page lock. However, MMU notifiers are * the page lock. However, MMU notifiers are
...@@ -656,6 +656,7 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt, ...@@ -656,6 +656,7 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt,
* be removed. * be removed.
*/ */
set_page_dirty(head_page); set_page_dirty(head_page);
}
/* on demand pinning support */ /* on demand pinning support */
if (!umem->context->invalidate_range) if (!umem->context->invalidate_range)
put_page(page); put_page(page);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment