• Ahmed S. Darwish's avatar
    mm/swap: Do not abuse the seqcount_t latching API · 6446a513
    Ahmed S. Darwish authored
    Commit eef1a429 ("mm/swap.c: piggyback lru_add_drain_all() calls")
    implemented an optimization mechanism to exit the to-be-started LRU
    drain operation (name it A) if another drain operation *started and
    finished* while (A) was blocked on the LRU draining mutex.
    
    This was done through a seqcount_t latch, which is an abuse of its
    semantics:
    
      1. seqcount_t latching should be used for the purpose of switching
         between two storage places with sequence protection to allow
         interruptible, preemptible, writer sections. The referenced
         optimization mechanism has absolutely nothing to do with that.
    
      2. The used raw_write_seqcount_latch() has two SMP write memory
         barriers to insure one consistent storage place out of the two
         storage places available. A full memory barrier is required
         instead: to guarantee that the pagevec counter stores visible by
         local CPU are visible to other CPUs -- before loading the current
         drain generation.
    
    Beside the seqcount_t API abuse, the semantics of a latch sequence
    counter was force-fitted into the referenced optimization. What was
    meant is to track "generations" of LRU draining operations, where
    "global lru draining generation = x" implies that all generations
    0 < n <= x are already *scheduled* for draining -- thus nothing needs
    to be done if the current generation number n <= x.
    
    Remove the conceptually-inappropriate seqcount_t latch usage. Manually
    implement the referenced optimization using a counter and SMP memory
    barriers.
    
    Note, while at it, use the non-atomic variant of cpumask_set_cpu(),
    __cpumask_set_cpu(), due to the already existing mutex protection.
    Signed-off-by: default avatarAhmed S. Darwish <a.darwish@linutronix.de>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Link: https://lkml.kernel.org/r/87y2pg9erj.fsf@vostro.fn.ogness.net
    6446a513
swap.c 34 KB