Commit 25ff1195 authored by Chris Wilson's avatar Chris Wilson Committed by Daniel Vetter

drm/i915: Workaround incoherence between fences and LLC across multiple CPUs

In order to fully serialize access to the fenced region and the update
to the fence register we need to take extreme measures on SNB+, and
manually flush writes to memory prior to writing the fence register in
conjunction with the memory barriers placed around the register write.

Fixes i-g-t/gem_fence_thrash

v2: Bring a bigger gun
v3: Switch the bigger gun for heavier bullets (Arjan van de Ven)
v4: Remove changes for working generations.
v5: Reduce to a per-cpu wbinvd() call prior to updating the fences.
v6: Rewrite comments to ellide forgotten history.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=62191Signed-off-by: default avatarChris Wilson <chris@chris-wilson.co.uk>
Cc: Jon Bloomfield <jon.bloomfield@intel.com>
Tested-by: Jon Bloomfield <jon.bloomfield@intel.com> (v2)
Cc: stable@vger.kernel.org
Reviewed-by: default avatarJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: default avatarDaniel Vetter <daniel.vetter@ffwll.ch>
parent 8bb6e959
...@@ -2683,17 +2683,35 @@ static inline int fence_number(struct drm_i915_private *dev_priv, ...@@ -2683,17 +2683,35 @@ static inline int fence_number(struct drm_i915_private *dev_priv,
return fence - dev_priv->fence_regs; return fence - dev_priv->fence_regs;
} }
static void i915_gem_write_fence__ipi(void *data)
{
wbinvd();
}
static void i915_gem_object_update_fence(struct drm_i915_gem_object *obj, static void i915_gem_object_update_fence(struct drm_i915_gem_object *obj,
struct drm_i915_fence_reg *fence, struct drm_i915_fence_reg *fence,
bool enable) bool enable)
{ {
struct drm_i915_private *dev_priv = obj->base.dev->dev_private; struct drm_device *dev = obj->base.dev;
int reg = fence_number(dev_priv, fence); struct drm_i915_private *dev_priv = dev->dev_private;
int fence_reg = fence_number(dev_priv, fence);
i915_gem_write_fence(obj->base.dev, reg, enable ? obj : NULL);
/* In order to fully serialize access to the fenced region and
* the update to the fence register we need to take extreme
* measures on SNB+. In theory, the write to the fence register
* flushes all memory transactions before, and coupled with the
* mb() placed around the register write we serialise all memory
* operations with respect to the changes in the tiler. Yet, on
* SNB+ we need to take a step further and emit an explicit wbinvd()
* on each processor in order to manually flush all memory
* transactions before updating the fence register.
*/
if (HAS_LLC(obj->base.dev))
on_each_cpu(i915_gem_write_fence__ipi, NULL, 1);
i915_gem_write_fence(dev, fence_reg, enable ? obj : NULL);
if (enable) { if (enable) {
obj->fence_reg = reg; obj->fence_reg = fence_reg;
fence->obj = obj; fence->obj = obj;
list_move_tail(&fence->lru_list, &dev_priv->mm.fence_list); list_move_tail(&fence->lru_list, &dev_priv->mm.fence_list);
} else { } else {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment