• Akinobu Mita's avatar
    blk-mq: avoid inserting requests before establishing new mapping · 5778322e
    Akinobu Mita authored
    Notifier callbacks for CPU_ONLINE action can be run on the other CPU
    than the CPU which was just onlined.  So it is possible for the
    process running on the just onlined CPU to insert request and run
    hw queue before establishing new mapping which is done by
    blk_mq_queue_reinit_notify().
    
    This can cause a problem when the CPU has just been onlined first time
    since the request queue was initialized.  At this time ctx->index_hw
    for the CPU, which is the index in hctx->ctxs[] for this ctx, is still
    zero before blk_mq_queue_reinit_notify() is called by notifier
    callbacks for CPU_ONLINE action.
    
    For example, there is a single hw queue (hctx) and two CPU queues
    (ctx0 for CPU0, and ctx1 for CPU1).  Now CPU1 is just onlined and
    a request is inserted into ctx1->rq_list and set bit0 in pending
    bitmap as ctx1->index_hw is still zero.
    
    And then while running hw queue, flush_busy_ctxs() finds bit0 is set
    in pending bitmap and tries to retrieve requests in
    hctx->ctxs[0]->rq_list.  But htx->ctxs[0] is a pointer to ctx0, so the
    request in ctx1->rq_list is ignored.
    
    Fix it by ensuring that new mapping is established before onlined cpu
    starts running.
    Signed-off-by: default avatarAkinobu Mita <akinobu.mita@gmail.com>
    Reviewed-by: default avatarMing Lei <tom.leiming@gmail.com>
    Cc: Jens Axboe <axboe@kernel.dk>
    Cc: Ming Lei <tom.leiming@gmail.com>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Signed-off-by: default avatarJens Axboe <axboe@fb.com>
    5778322e
blk-mq.h 3.38 KB