• Monk Liu's avatar
    drm/amdgpu: refactoring mailbox to fix TDR handshake bugs(v2) · 48527e52
    Monk Liu authored
    this patch actually refactor mailbox implmentations, and
    all below changes are needed together to fix all those mailbox
    handshake issues exposured by heavey TDR test.
    
    1)refactor all mailbox functions based on byte accessing for mb_control
    reason is to avoid touching non-related bits when writing trn/rcv part of
    mailbox_control, this way some incorrect INTR sent to hypervisor
    side could be avoided, and it fixes couple handshake bug.
    
    2)trans_msg function re-impled: put a invalid
    logic before transmitting message to make sure the ACK bit is in
    a clear status, otherwise there is chance that ACK asserted already
    before transmitting message and lead to fake ACK polling.
    (hypervisor side have some tricks to workaround ACK bit being corrupted
    by VF FLR which hase an side effects that may make guest side ACK bit
    asserted wrongly), and clear TRANS_MSG words after message transferred.
    
    3)for mailbox_flr_work, it is also re-worked: it takes the mutex lock
    first if invoked, to block gpu recover's participate too early while
    hypervisor side is doing VF FLR. (hypervisor sends FLR_NOTIFY to guest
    before doing VF FLR and sentds FLR_COMPLETE after VF FLR done, and
    the FLR_NOTIFY will trigger interrupt to guest which lead to
    mailbox_flr_work being invoked)
    
    This can avoid the issue that mailbox trans msg being cleared by its VF FLR.
    
    4)for mailbox_rcv_irq IRQ routine, it should only peek msg and schedule
    mailbox_flr_work, instead of ACK to hypervisor itself, because FLR_NOTIFY
    msg sent from hypervisor side doesn't need VF's ACK (this is because
    VF's ACK would lead to hypervisor clear its trans_valid/msg, and this
    would cause handshake bug if trans_valid/msg is cleared not due to
    correct VF ACK but from a wrong VF ACK like this "FLR_NOTIFY" one)
    
    This fixed handshake bug that sometimes GUEST always couldn't receive
    "READY_TO_ACCESS_GPU" msg from hypervisor.
    
    5)seperate polling time limite accordingly:
    POLL ACK cost no more than 500ms
    POLL MSG cost no more than 12000ms
    POLL FLR finish cost no more than 500ms
    
    6) we still need to set adev into in_gpu_reset mode after we received
    FLR_NOTIFY from host side, this can prevent innocent app wrongly succesed
    to open amdgpu dri device.
    
    FLR_NOFITY is received due to an IDLE hang detected from hypervisor side
    which indicating GPU is already die in this VF.
    
    v2:
    use MACRO as the offset of mailbox_control register
    don't test if NOTIFY_CMPL event in rcv_msg since it won't
    recieve that message anymore
    Signed-off-by: default avatarMonk Liu <Monk.Liu@amd.com>
    Reviewed-by: default avatarPixel Ding <Pixel.Ding@amd.com>
    Signed-off-by: default avatarAlex Deucher <alexander.deucher@amd.com>
    48527e52
mxgpu_ai.c 10.2 KB