• Brett Creeley's avatar
    ice: Refactor interrupt tracking · cbe66bfe
    Brett Creeley authored
    Currently we have two MSI-x (IRQ) trackers, one for OS requested MSI-x
    entries (sw_irq_tracker) and one for hardware MSI-x vectors
    (hw_irq_tracker). Generally the sw_irq_tracker has less entries than the
    hw_irq_tracker because the hw_irq_tracker has entries equal to the max
    allowed MSI-x per PF and the sw_irq_tracker is mainly the minimum (non
    SR-IOV portion of the vectors, kernel granted IRQs). All of the non
    SR-IOV portions of the driver (i.e. LAN queues, RDMA queues, OICR, etc.)
    take at least one of each type of tracker resource. SR-IOV only grabs
    entries from the hw_irq_tracker. There are a few issues with this approach
    that can be seen when doing any kind of device reconfiguration (i.e.
    ethtool -L, SR-IOV, etc.). One of them being, any time the driver creates
    an ice_q_vector and associates it to a LAN queue pair it will grab and
    use one entry from the hw_irq_tracker and one from the sw_irq_tracker.
    If the indices on these does not match it will cause a Tx timeout, which
    will cause a reset and then the indices will match up again and traffic
    will resume. The mismatched indices come from the trackers not being the
    same size and/or the search_hint in the two trackers not being equal.
    Another reason for the refactor is the co-existence of features with
    SR-IOV. If SR-IOV is enabled and the interrupts are taken from the end
    of the sw_irq_tracker then other features can no longer use this space
    because the hardware has now given the remaining interrupts to SR-IOV.
    
    This patch reworks how we track MSI-x vectors by removing the
    hw_irq_tracker completely and instead MSI-x resources needed for SR-IOV
    are determined all at once instead of per VF. This can be done because
    when creating VFs we know how many are wanted and how many MSI-x vectors
    each VF needs. This also allows us to start using MSI-x resources from
    the end of the PF's allowed MSI-x vectors so we are less likely to use
    entries needed for other features (i.e. RDMA, L2 Offload, etc).
    
    This patch also reworks the ice_res_tracker structure by removing the
    search_hint and adding a new member - "end". Instead of having a
    search_hint we will always search from 0. The new member, "end", will be
    used to manipulate the end of the ice_res_tracker (specifically
    sw_irq_tracker) during runtime based on MSI-x vectors needed by SR-IOV.
    In the normal case, the end of ice_res_tracker will be equal to the
    ice_res_tracker's num_entries.
    
    The sriov_base_vector member was added to the PF structure. It is used
    to represent the starting MSI-x index of all the needed MSI-x vectors
    for all SR-IOV VFs. Depending on how many MSI-x are needed, SR-IOV may
    have to take resources from the sw_irq_tracker. This is done by setting
    the sw_irq_tracker->end equal to the pf->sriov_base_vector. When all
    SR-IOV VFs are removed then the sw_irq_tracker->end is reset back to
    sw_irq_tracker->num_entries. The sriov_base_vector, along with the VF's
    number of MSI-x (pf->num_vf_msix), vf_id, and the base MSI-x index on
    the PF (pf->hw.func_caps.common_cap.msix_vector_first_id), is used to
    calculate the first HW absolute MSI-x index for each VF, which is used
    to write to the VPINT_ALLOC[_PCI] and GLINT_VECT2FUNC registers to
    program the VFs MSI-x PCI configuration bits. Also, the sriov_base_vector
    is used along with VF's num_vf_msix, vf_id, and q_vector->v_idx to
    determine the MSI-x register index (used for writing to GLINT_DYN_CTL)
    within the PF's space.
    
    Interrupt changes removed any references to hw_base_vector, hw_oicr_idx,
    and hw_irq_tracker. Only sw_base_vector, sw_oicr_idx, and sw_irq_tracker
    variables remain. Change all of these by removing the "sw_" prefix to
    help avoid confusion with these variables and their use.
    Signed-off-by: default avatarBrett Creeley <brett.creeley@intel.com>
    Signed-off-by: default avatarAnirudh Venkataramanan <anirudh.venkataramanan@intel.com>
    Tested-by: default avatarAndrew Bowers <andrewx.bowers@intel.com>
    Signed-off-by: default avatarJeff Kirsher <jeffrey.t.kirsher@intel.com>
    cbe66bfe
ice_ethtool.c 94.3 KB