Commit 16c11325 authored by Lorenzo Pieralisi's avatar Lorenzo Pieralisi Committed by Catalin Marinas

arm64: mm: change IOMMU notifier action to attach DMA ops

Current bus notifier in ARM64 (__iommu_attach_notifier)
attempts to attach dma_ops to a device on BUS_NOTIFY_ADD_DEVICE
action notification.

This will cause issues on ACPI based systems, where PCI devices
can be added before the IOMMUs the devices are attached to
had a chance to be probed, causing failures on attempts to
attach dma_ops in that the domain for the respective IOMMU
may not be set-up yet by the time the bus notifier is run.

Devices dma_ops do not require to be set-up till the matching
device drivers are probed. This means that instead of running
the notifier attaching dma_ops to devices (__iommu_attach_notifier)
on BUS_NOTIFY_ADD_DEVICE action, it can be run just before the
device driver is bound to the device in question (on action
BUS_NOTIFY_BIND_DRIVER) so that it is certain that its IOMMU
group and domain are set-up accordingly at the time the
notifier is triggered.

This patch changes the notifier action upon which dma_ops
are attached to devices and defer it to driver binding time,
so that IOMMU devices have a chance to be probed and to register
their bus notifiers before the dma_ops attach sequence for a
device is actually carried out.

As a result we also no longer need worry about racing with
iommu_bus_notifier(), or about retrying the queue in case devices
were added too early on DT-based systems, so clean up the notifier
itself plus the additional workaround from 722ec35f ("arm64:
dma-mapping: fix handling of devices registered before arch_initcall")
Acked-by: default avatarWill Deacon <will.deacon@arm.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: default avatarLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
[rm: get rid of other now-redundant bits]
Signed-off-by: default avatarRobin Murphy <robin.murphy@arm.com>
Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
parent 19a469a5
...@@ -861,15 +861,16 @@ static int __iommu_attach_notifier(struct notifier_block *nb, ...@@ -861,15 +861,16 @@ static int __iommu_attach_notifier(struct notifier_block *nb,
{ {
struct iommu_dma_notifier_data *master, *tmp; struct iommu_dma_notifier_data *master, *tmp;
if (action != BUS_NOTIFY_ADD_DEVICE) if (action != BUS_NOTIFY_BIND_DRIVER)
return 0; return 0;
mutex_lock(&iommu_dma_notifier_lock); mutex_lock(&iommu_dma_notifier_lock);
list_for_each_entry_safe(master, tmp, &iommu_dma_masters, list) { list_for_each_entry_safe(master, tmp, &iommu_dma_masters, list) {
if (do_iommu_attach(master->dev, master->ops, if (data == master->dev && do_iommu_attach(master->dev,
master->dma_base, master->size)) { master->ops, master->dma_base, master->size)) {
list_del(&master->list); list_del(&master->list);
kfree(master); kfree(master);
break;
} }
} }
mutex_unlock(&iommu_dma_notifier_lock); mutex_unlock(&iommu_dma_notifier_lock);
...@@ -883,17 +884,8 @@ static int __init register_iommu_dma_ops_notifier(struct bus_type *bus) ...@@ -883,17 +884,8 @@ static int __init register_iommu_dma_ops_notifier(struct bus_type *bus)
if (!nb) if (!nb)
return -ENOMEM; return -ENOMEM;
/*
* The device must be attached to a domain before the driver probe
* routine gets a chance to start allocating DMA buffers. However,
* the IOMMU driver also needs a chance to configure the iommu_group
* via its add_device callback first, so we need to make the attach
* happen between those two points. Since the IOMMU core uses a bus
* notifier with default priority for add_device, do the same but
* with a lower priority to ensure the appropriate ordering.
*/
nb->notifier_call = __iommu_attach_notifier; nb->notifier_call = __iommu_attach_notifier;
nb->priority = -100;
ret = bus_register_notifier(bus, nb); ret = bus_register_notifier(bus, nb);
if (ret) { if (ret) {
...@@ -917,10 +909,6 @@ static int __init __iommu_dma_init(void) ...@@ -917,10 +909,6 @@ static int __init __iommu_dma_init(void)
if (!ret) if (!ret)
ret = register_iommu_dma_ops_notifier(&pci_bus_type); ret = register_iommu_dma_ops_notifier(&pci_bus_type);
#endif #endif
/* handle devices queued before this arch_initcall */
if (!ret)
__iommu_attach_notifier(NULL, BUS_NOTIFY_ADD_DEVICE, NULL);
return ret; return ret;
} }
arch_initcall(__iommu_dma_init); arch_initcall(__iommu_dma_init);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment