- 13 Dec, 2018 15 commits
-
-
Christoph Hellwig authored
While the dma-direct code is (relatively) clean and simple we actually have to use the swiotlb ops for the mapping on many architectures due to devices with addressing limits. Instead of keeping two implementations around this commit allows the dma-direct implementation to call the swiotlb bounce buffering functions and thus share the guts of the mapping implementation. This also simplified the dma-mapping setup on a few architectures where we don't have to differenciate which implementation to use. Signed-off-by:
Christoph Hellwig <hch@lst.de> Acked-by:
Jesper Dangaard Brouer <brouer@redhat.com> Tested-by:
Tony Luck <tony.luck@intel.com>
-
Christoph Hellwig authored
No need to duplicate the mapping logic. Signed-off-by:
-
Christoph Hellwig authored
Only report report a DMA addressability report once to avoid spewing the kernel log with repeated message. Also provide a stack trace to make it easy to find the actual caller that caused the problem. Last but not least move the actual check into the fast path and only leave the error reporting in a helper. Signed-off-by:
-
Christoph Hellwig authored
Instead of providing a special dma_mark_clean hook just for ia64, switch ia64 to use the normal arch_sync_dma_for_cpu hooks instead. This means that we now also set the PG_arch_1 bit for pages in the swiotlb buffer, which isn't stricly needed as we will never execute code out of the swiotlb buffer, but otherwise harmless. Signed-off-by:
-
Christoph Hellwig authored
We can use DMA_MAPPING_ERROR instead, which already maps to the same value. Signed-off-by:
-
Robin Murphy authored
Rather than checking the DMA attribute at each callsite, just pass it through for acpi_dma_configure() to handle directly. That can then deal with the relatively exceptional DEV_DMA_NOT_SUPPORTED case by explicitly installing dummy DMA ops instead of just skipping setup entirely. This will then free up the dev->dma_ops == NULL case for some valuable fastpath optimisations. Signed-off-by:
Robin Murphy <robin.murphy@arm.com> Reviewed-by:
Rafael J. Wysocki <rafael.j.wysocki@intel.com> Acked-by:
-
Robin Murphy authored
The dummy DMA ops are currently used by arm64 for any device which has an invalid ACPI description and is thus barred from using DMA due to not knowing whether is is cache-coherent or not. Factor these out into general dma-mapping code so that they can be referenced from other common code paths. In the process, we can prune all the optional callbacks which just do the same thing as the default behaviour, and fill in .map_resource for completeness. Signed-off-by:
-
Christoph Hellwig authored
All architectures except for sparc64 use the dma-direct code in some form, and even for sparc64 we had the discussion of a direct mapping mode a while ago. In preparation for directly calling the direct mapping code don't bother having it optionally but always build the code in. This is a minor hardship for some powerpc and arm configs that don't pull it in yet (although they should in a relase ot two), and sparc64 which currently doesn't need it at all, but it will reduce the ifdef mess we'd otherwise need significantly. Signed-off-by:
-
Christoph Hellwig authored
This isn't exactly a slow path routine, but it is not super critical either, and moving it out of line will help to keep the include chain clean for the following DMA indirection bypass work. Signed-off-by:
-
Christoph Hellwig authored
There is no need to have all setup and coherent allocation / freeing routines inline. Move them out of line to keep the implemeation nicely encapsulated and save some kernel text size. Signed-off-by:
-
Christoph Hellwig authored
dma_get_required_mask should really be with the rest of the DMA mapping implementation instead of in drivers/base as a lone outlier. Signed-off-by:
-
Christoph Hellwig authored
The two functions are exactly the same, so don't bother implementing them twice. Signed-off-by:
-
Christoph Hellwig authored
We can just call the regular calls after adding offset the the address instead of reimplementing them. Signed-off-by:
-
Christoph Hellwig authored
We already zero the memory after allocating it from the pool that this function fills, and having the memset here in this form means we can't support CMA highmem allocations. Signed-off-by:
Russell King - ARM Linux <linux@armlinux.org.uk>
-
Christoph Hellwig authored
The sparc tree already has this change for the pre-refactored code, but pulling it into the dma-mapping tree like this should ease the merge conflicts a bit. Signed-off-by:
Sam Ravnborg <sam@ravnborg.org> Acked-by:
David Miller <davem@davemloft.net>
-
- 11 Dec, 2018 13 commits
-
-
Christoph Hellwig authored
There are enough common defintions that a single header seems nicer. Also drop the pointless <linux/dma-mapping.h> include. Signed-off-by:
-
Christoph Hellwig authored
It has nothing to do with the content of the pci.h header. Suggested by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by:
-
Christoph Hellwig authored
The only thing we need to explicitly pull in is the defines for the CPU type. Signed-off-by:
-
Christoph Hellwig authored
There is no good reason to have a double indirection for the sparc32 dma ops, so remove the sparc32_dma_ops and define separate dma_map_ops instance for the different IOMMU types. Signed-off-by:
-
Christoph Hellwig authored
Factor the code to remap memory returned from the DMA coherent allocator into two helpers that can be shared by the IOMMU and direct mapping code. Signed-off-by:
-
Christoph Hellwig authored
No need to BUG_ON() on the cache maintainance ops - they are no-ops by default, and there is nothing in the DMA API contract that prohibits calling them on sbus devices (even if such drivers are unlikely to ever appear). Similarly a dma_supported method that always returns 0 is rather pointless. The only thing that indicates is that no one ever calls the method on sbus devices. Signed-off-by:
-
Robin Murphy authored
DMA debug entries are one of those things which aren't that useful individually - we will always want some larger quantity of them - and which we don't really need to manage the exact number of - we only care about having 'enough'. In that regard, the current behaviour of creating them one-by-one leads to a lot of unwarranted function call overhead and memory wasted on alignment padding. Now that we don't have to worry about freeing anything via dma_debug_resize_entries(), we can optimise the allocation behaviour by grabbing whole pages at once, which will save considerably on the aforementioned overheads, and probably offer a little more cache/TLB locality benefit for traversing the lists under normal operation. This should also give even less reason for an architecture-level override of the preallocation size, so make the definition unconditional - if there is still any desire to change the compile-time value for some platforms it would be better off as a Kconfig option anyway. Since freeing a whole page of entries at once becomes enough of a challenge that it's not really worth complicating dma_debug_init(), we may as well tweak the preallocation behaviour such that as long as we manage to allocate *some* pages, we can leave debugging enabled on a best-effort basis rather than otherwise wasting them. Signed-off-by:
Qian Cai <cai@lca.pw> Signed-off-by:
-
Robin Murphy authored
With the only caller now gone, we can clean up this part of dma-debug's exposed internals and make way to tweak the allocation behaviour. Signed-off-by:
-
Robin Murphy authored
dma-debug is now capable of adding new entries to its pool on-demand if the initial preallocation was insufficient, so the IOMMU_LEAK logic no longer needs to explicitly change the pool size. This does lose it the ability to save a couple of megabytes of RAM by reducing the pool size below its default, but it seems unlikely that that is a realistic concern these days (or indeed that anyone is actively debugging AGP drivers' DMA usage any more). Getting rid of dma_debug_resize_entries() will make room for further streamlining in the dma-debug code itself. Removing the call reveals quite a lot of cruft which has been useless for nearly a decade since commit 19c1a6f5 ("x86 gart: reimplement IOMMU_LEAK feature by using DMA_API_DEBUG"), including the entire 'iommu=leak' parameter, which controlled nothing except whether dma_debug_resize_entries() was called or not. Signed-off-by:
Thomas Gleixner <tglx@linutronix.de> Tested-by:
-
Robin Murphy authored
Now that we can dynamically allocate DMA debug entries to cope with drivers maintaining excessively large numbers of live mappings, a driver which *does* actually have a bug leaking mappings (and is not unloaded) will no longer trigger the "DMA-API: debugging out of memory - disabling" message until it gets to actual kernel OOM conditions, which means it could go unnoticed for a while. To that end, let's inform the user each time the pool has grown to a multiple of its initial size, which should make it apparent that they either have a leak or might want to increase the preallocation size. Signed-off-by:
-
Robin Murphy authored
Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option. Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time their code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure, short of massively over-allocating. There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour. To that end, refactor the prealloc_memory() logic a little bit to generalise it for runtime reallocations as well. Signed-off-by:
-
Robin Murphy authored
Use pr_fmt() to generate the "DMA-API: " prefix consistently. This results in it being added to a couple of pr_*() messages which were missing it before, and for the err_printk() calls moves it to the actual start of the message instead of somewhere in the middle. Signed-off-by:
-
Robin Murphy authored
Expose nr_total_entries in debugfs, so that {num,min}_free_entries become even more meaningful to users interested in current/maximum utilisation. This becomes even more relevant once nr_total_entries may change at runtime beyond just the existing AMD GART debug code. Suggested-by:John Garry <john.garry@huawei.com> Signed-off-by:
-
- 06 Dec, 2018 12 commits
-
-
Christoph Hellwig authored
These days architectures are mostly out of the business of dealing with struct scatterlist at all, unless they have architecture specific iommu drivers. Replace the ARCH_HAS_SG_CHAIN symbol with a ARCH_NO_SG_CHAIN one only enabled for architectures with horrible legacy iommu drivers like alpha and parisc, and conditionally for arm which wants to keep it disable for legacy platforms. Signed-off-by:
Palmer Dabbelt <palmer@sifive.com>
-
Christoph Hellwig authored
Currently dma_mapping_error returns a boolean as int, with 1 meaning error. This is rather unusual and many callers have to convert it to errno value. The callers are highly inconsistent with error codes ranging from -ENOMEM over -EIO, -EINVAL and -EFAULT ranging to -EAGAIN. Return -ENOMEM which seems to be what the largest number of callers convert it to, and which also matches the typical error case where we are out of resources. Signed-off-by:
Russell King <rmk+kernel@armlinux.org.uk> Acked-by:
Linus Torvalds <torvalds@linux-foundation.org>
-
Christoph Hellwig authored
No users left except for vmd which just forwards it. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Signed-off-by:
-
Christoph Hellwig authored
Pass the page + offset to the low-level __iommu_map_single helper (which gets renamed to fit the new calling conventions) as both callers have the page at hand. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Note that the existing code used AMD_IOMMU_MAPPING_ERROR to check from a 0 return from the IOVA allocator, which is replaced with an explicit 0 as in the implementation and other users of that interface. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of the magic bad_dma_addr on a dma mapping failure and let the core dma-mapping code handle the rest. Remove the magic EMERGENCY_PAGES that the bad_dma_addr gets redirected to. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of the magic bad_dma_addr on a dma mapping failure and let the core dma-mapping code handle the rest. Remove the magic EMERGENCY_PAGES that the bad_dma_addr gets redirected to. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Signed-off-by:
-
Christoph Hellwig authored
Return DMA_MAPPING_ERROR instead of 0 on a dma mapping failure and let the core dma-mapping code handle the rest. Signed-off-by:
-
