• Will Deacon's avatar
    swiotlb: Fix double-allocation of slots due to broken alignment handling · 04867a7a
    Will Deacon authored
    Commit bbb73a10 ("swiotlb: fix a braino in the alignment check fix"),
    which was a fix for commit 0eee5ae1 ("swiotlb: fix slot alignment
    checks"), causes a functional regression with vsock in a virtual machine
    using bouncing via a restricted DMA SWIOTLB pool.
    
    When virtio allocates the virtqueues for the vsock device using
    dma_alloc_coherent(), the SWIOTLB search can return page-unaligned
    allocations if 'area->index' was left unaligned by a previous allocation
    from the buffer:
    
     # Final address in brackets is the SWIOTLB address returned to the caller
     | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1645-1649/7168 (0x98326800)
     | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1649-1653/7168 (0x98328800)
     | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1653-1657/7168 (0x9832a800)
    
    This ends badly (typically buffer corruption and/or a hang) because
    swiotlb_alloc() is expecting a page-aligned allocation and so blindly
    returns a pointer to the 'struct page' corresponding to the allocation,
    therefore double-allocating the first half (2KiB slot) of the 4KiB page.
    
    Fix the problem by treating the allocation alignment separately to any
    additional alignment requirements from the device, using the maximum
    of the two as the stride to search the buffer slots and taking care
    to ensure a minimum of page-alignment for buffers larger than a page.
    
    This also resolves swiotlb allocation failures occuring due to the
    inclusion of ~PAGE_MASK in 'iotlb_align_mask' for large allocations and
    resulting in alignment requirements exceeding swiotlb_max_mapping_size().
    
    Fixes: bbb73a10 ("swiotlb: fix a braino in the alignment check fix")
    Fixes: 0eee5ae1 ("swiotlb: fix slot alignment checks")
    Signed-off-by: default avatarWill Deacon <will@kernel.org>
    Reviewed-by: default avatarMichael Kelley <mhklinux@outlook.com>
    Reviewed-by: default avatarPetr Tesarik <petr.tesarik1@huawei-partners.com>
    Tested-by: default avatarNicolin Chen <nicolinc@nvidia.com>
    Tested-by: default avatarMichael Kelley <mhklinux@outlook.com>
    Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    04867a7a
swiotlb.c 48.8 KB