1. 24 Mar, 2014 27 commits
  2. 20 Mar, 2014 3 commits
  3. 19 Mar, 2014 4 commits
    • David Woodhouse's avatar
      iommu/vt-d: Be less pessimistic about domain coherency where possible · d0501960
      David Woodhouse authored
      In commit 2e12bc29 ("intel-iommu: Default to non-coherent for domains
      unattached to iommus") we decided to err on the side of caution and
      always assume that it's possible that a device will be attached which is
      behind a non-coherent IOMMU.
      
      In some cases, however, that just *cannot* happen. If there *are* no
      IOMMUs in the system which are non-coherent, then we don't need to do
      it. And flushing the dcache is a *significant* performance hit.
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      d0501960
    • David Woodhouse's avatar
    • David Woodhouse's avatar
      iommu/vt-d: Clean up and fix page table clear/free behaviour · ea8ea460
      David Woodhouse authored
      There is a race condition between the existing clear/free code and the
      hardware. The IOMMU is actually permitted to cache the intermediate
      levels of the page tables, and doesn't need to walk the table from the
      very top of the PGD each time. So the existing back-to-back calls to
      dma_pte_clear_range() and dma_pte_free_pagetable() can lead to a
      use-after-free where the IOMMU reads from a freed page table.
      
      When freeing page tables we actually need to do the IOTLB flush, with
      the 'invalidation hint' bit clear to indicate that it's not just a
      leaf-node flush, after unlinking each page table page from the next level
      up but before actually freeing it.
      
      So in the rewritten domain_unmap() we just return a list of pages (using
      pg->freelist to make a list of them), and then the caller is expected to
      do the appropriate IOTLB flush (or tear down the domain completely,
      whatever), before finally calling dma_free_pagelist() to free the pages.
      
      As an added bonus, we no longer need to flush the CPU's data cache for
      pages which are about to be *removed* from the page table hierarchy anyway,
      in the non-cache-coherent case. This drastically improves the performance
      of large unmaps.
      
      As a side-effect of all these changes, this also fixes the fact that
      intel_iommu_unmap() was neglecting to free the page tables for the range
      in question after clearing them.
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      ea8ea460
    • David Woodhouse's avatar
      iommu/vt-d: Clean up size handling for intel_iommu_unmap() · 5cf0a76f
      David Woodhouse authored
      We have this horrid API where iommu_unmap() can unmap more than it's asked
      to, if the IOVA in question happens to be mapped with a large page.
      
      Instead of propagating this nonsense to the point where we end up returning
      the page order from dma_pte_clear_range(), let's just do it once and adjust
      the 'size' parameter accordingly.
      
      Augment pfn_to_dma_pte() to return the level at which the PTE was found,
      which will also be useful later if we end up changing the API for
      iommu_iova_to_phys() to behave the same way as is being discussed upstream.
      Signed-off-by: default avatarDavid Woodhouse <David.Woodhouse@intel.com>
      5cf0a76f
  4. 04 Mar, 2014 6 commits
    • Jiang Liu's avatar
      iommu/vt-d: Update IOMMU state when memory hotplug happens · 75f05569
      Jiang Liu authored
      If static identity domain is created, IOMMU driver needs to update
      si_domain page table when memory hotplug event happens. Otherwise
      PCI device DMA operations can't access the hot-added memory regions.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      75f05569
    • Jiang Liu's avatar
      iommu/vt-d: Unify the way to process DMAR device scope array · 2e455289
      Jiang Liu authored
      Now we have a PCI bus notification based mechanism to update DMAR
      device scope array, we could extend the mechanism to support boot
      time initialization too, which will help to unify and simplify
      the implementation.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      2e455289
    • Jiang Liu's avatar
      iommu/vt-d: Update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens · 59ce0515
      Jiang Liu authored
      Current Intel DMAR/IOMMU driver assumes that all PCI devices associated
      with DMAR/RMRR/ATSR device scope arrays are created at boot time and
      won't change at runtime, so it caches pointers of associated PCI device
      object. That assumption may be wrong now due to:
      1) introduction of PCI host bridge hotplug
      2) PCI device hotplug through sysfs interfaces.
      
      Wang Yijing has tried to solve this issue by caching <bus, dev, func>
      tupple instead of the PCI device object pointer, but that's still
      unreliable because PCI bus number may change in case of hotplug.
      Please refer to http://lkml.org/lkml/2013/11/5/64
      Message from Yingjing's mail:
      after remove and rescan a pci device
      [  611.857095] dmar: DRHD: handling fault status reg 2
      [  611.857109] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff7000
      [  611.857109] DMAR:[fault reason 02] Present bit in context entry is clear
      [  611.857524] dmar: DRHD: handling fault status reg 102
      [  611.857534] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff6000
      [  611.857534] DMAR:[fault reason 02] Present bit in context entry is clear
      [  611.857936] dmar: DRHD: handling fault status reg 202
      [  611.857947] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff5000
      [  611.857947] DMAR:[fault reason 02] Present bit in context entry is clear
      [  611.858351] dmar: DRHD: handling fault status reg 302
      [  611.858362] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff4000
      [  611.858362] DMAR:[fault reason 02] Present bit in context entry is clear
      [  611.860819] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
      [  611.860983] dmar: DRHD: handling fault status reg 402
      [  611.860995] dmar: INTR-REMAP: Request device [[86:00.3] fault index a4
      [  611.860995] INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
      
      This patch introduces a new mechanism to update the DRHD/RMRR/ATSR device scope
      caches by hooking PCI bus notification.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      59ce0515
    • Jiang Liu's avatar
      iommu/vt-d: Use RCU to protect global resources in interrupt context · 0e242612
      Jiang Liu authored
      Global DMA and interrupt remapping resources may be accessed in
      interrupt context, so use RCU instead of rwsem to protect them
      in such cases.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      0e242612
    • Jiang Liu's avatar
      iommu/vt-d: Introduce a rwsem to protect global data structures · 3a5670e8
      Jiang Liu authored
      Introduce a global rwsem dmar_global_lock, which will be used to
      protect DMAR related global data structures from DMAR/PCI/memory
      device hotplug operations in process context.
      
      DMA and interrupt remapping related data structures are read most,
      and only change when memory/PCI/DMAR hotplug event happens.
      So a global rwsem solution is adopted for balance between simplicity
      and performance.
      
      For interrupt remapping driver, function intel_irq_remapping_supported(),
      dmar_table_init(), intel_enable_irq_remapping(), disable_irq_remapping(),
      reenable_irq_remapping() and enable_drhd_fault_handling() etc
      are called during booting, suspending and resuming with interrupt
      disabled, so no need to take the global lock.
      
      For interrupt remapping entry allocation, the locking model is:
      	down_read(&dmar_global_lock);
      	/* Find corresponding iommu */
      	iommu = map_hpet_to_ir(id);
      	if (iommu)
      		/*
      		 * Allocate remapping entry and mark entry busy,
      		 * the IOMMU won't be hot-removed until the
      		 * allocated entry has been released.
      		 */
      		index = alloc_irte(iommu, irq, 1);
      	up_read(&dmar_global_lock);
      
      For DMA remmaping driver, we only uses the dmar_global_lock rwsem to
      protect functions which are only called in process context. For any
      function which may be called in interrupt context, we will use RCU
      to protect them in following patches.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      3a5670e8
    • Jiang Liu's avatar
      iommu/vt-d: Introduce macro for_each_dev_scope() to walk device scope entries · b683b230
      Jiang Liu authored
      Introduce for_each_dev_scope()/for_each_active_dev_scope() to walk
      {active} device scope entries. This will help following RCU lock
      related patches.
      Signed-off-by: default avatarJiang Liu <jiang.liu@linux.intel.com>
      Signed-off-by: default avatarJoerg Roedel <joro@8bytes.org>
      b683b230