-
Chris Wilson authored
Invalidate TLB in batches, in order to reduce performance regressions. Currently, every caller performs a full barrier around a TLB invalidation, ignoring all other invalidations that may have already removed their PTEs from the cache. As this is a synchronous operation and can be quite slow, we cause multiple threads to contend on the TLB invalidate mutex blocking userspace. We only need to invalidate the TLB once after replacing our PTE to ensure that there is no possible continued access to the physical address before releasing our pages. By tracking a seqno for each full TLB invalidate we can quickly determine if one has been performed since rewriting the PTE, and only if necessary trigger one for ourselves. That helps to reduce the performance regression introduced by TLB invalidate logic. [mchehab: rebased to not require moving the code to a separate file] Cc: stable@vger.kernel.org Fixes: 7938d615 ("drm/i915: Flush TLBs before releasing backing store") Suggested-by:
Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by:
Chris Wilson <chris.p.wilson@intel.com> Cc: Fei Yang <fei.yang@intel.com> Signed-off-by:
Mauro Carvalho Chehab <mchehab@kernel.org> Acked-by:
Andi Shyti <andi.shyti@linux.intel.com> Signed-off-by:
5d36acb7
