• Peter Zijlstra's avatar
    sched/topology: Fix overlapping sched_group_capacity · 1676330e
    Peter Zijlstra authored
    When building the overlapping groups we need to attach a consistent
    sched_group_capacity structure. That is, all 'identical' sched_group's
    should have the _same_ sched_group_capacity.
    
    This can (once again) be demonstrated with a topology like:
    
      node   0   1   2   3
        0:  10  20  30  20
        1:  20  10  20  30
        2:  30  20  10  20
        3:  20  30  20  10
    
    But we need at least 2 CPUs per node for this to show up, after all,
    if there is only one CPU per node, our CPU @i is per definition a
    unique CPU that reaches this domain (aka balance-cpu).
    
    Given the above NUMA topo and 2 CPUs per node:
    
      [] CPU0 attaching sched-domain(s):
      []  domain-0: span=0,4 level=DIE
      []   groups: 0:{ span=0 }, 4:{ span=4 }
      []   domain-1: span=0-1,3-5,7 level=NUMA
      []    groups: 0:{ span=0,4 mask=0,4 cap=2048 }, 1:{ span=1,5 mask=1,5 cap=2048 }, 3:{ span=3,7 mask=3,7 cap=2048 }
      []    domain-2: span=0-7 level=NUMA
      []     groups: 0:{ span=0-1,3-5,7 mask=0,4 cap=6144 }, 2:{ span=1-3,5-7 mask=2,6 cap=6144 }
      [] CPU1 attaching sched-domain(s):
      []  domain-0: span=1,5 level=DIE
      []   groups: 1:{ span=1 }, 5:{ span=5 }
      []   domain-1: span=0-2,4-6 level=NUMA
      []    groups: 1:{ span=1,5 mask=1,5 cap=2048 }, 2:{ span=2,6 mask=2,6 cap=2048 }, 4:{ span=0,4 mask=0,4 cap=2048 }
      []    domain-2: span=0-7 level=NUMA
      []     groups: 1:{ span=0-2,4-6 mask=1,5 cap=6144 }, 3:{ span=0,2-4,6-7 mask=3,7 cap=6144 }
    
    Observe how CPU0-domain1-group0 and CPU1-domain1-group4 are the
    'same' but have a different id (0 vs 4).
    
    To fix this, use the group balance CPU to select the SGC. This means
    we have to compute the full mask for each CPU and require a second
    temporary mask to store the group mask in (it otherwise lives in the
    SGC).
    
    The fixed topology looks like:
    
      [] CPU0 attaching sched-domain(s):
      []  domain-0: span=0,4 level=DIE
      []   groups: 0:{ span=0 }, 4:{ span=4 }
      []   domain-1: span=0-1,3-5,7 level=NUMA
      []    groups: 0:{ span=0,4 mask=0,4 cap=2048 }, 1:{ span=1,5 mask=1,5 cap=2048 }, 3:{ span=3,7 mask=3,7 cap=2048 }
      []    domain-2: span=0-7 level=NUMA
      []     groups: 0:{ span=0-1,3-5,7 mask=0,4 cap=6144 }, 2:{ span=1-3,5-7 mask=2,6 cap=6144 }
      [] CPU1 attaching sched-domain(s):
      []  domain-0: span=1,5 level=DIE
      []   groups: 1:{ span=1 }, 5:{ span=5 }
      []   domain-1: span=0-2,4-6 level=NUMA
      []    groups: 1:{ span=1,5 mask=1,5 cap=2048 }, 2:{ span=2,6 mask=2,6 cap=2048 }, 0:{ span=0,4 mask=0,4 cap=2048 }
      []    domain-2: span=0-7 level=NUMA
      []     groups: 1:{ span=0-2,4-6 mask=1,5 cap=6144 }, 3:{ span=0,2-4,6-7 mask=3,7 cap=6144 }
    Debugged-by: default avatarLauro Ramos Venancio <lvenanci@redhat.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Mike Galbraith <efault@gmx.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: linux-kernel@vger.kernel.org
    Fixes: e3589f6c ("sched: Allow for overlapping sched_domain spans")
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    1676330e
topology.c 40.9 KB