• Jiong Wang's avatar
    bpf: verifier: mark verified-insn with sub-register zext flag · 5327ed3d
    Jiong Wang authored
    eBPF ISA specification requires high 32-bit cleared when low 32-bit
    sub-register is written. This applies to destination register of ALU32 etc.
    JIT back-ends must guarantee this semantic when doing code-gen. x86_64 and
    AArch64 ISA has the same semantics, so the corresponding JIT back-end
    doesn't need to do extra work.
    
    However, 32-bit arches (arm, x86, nfp etc.) and some other 64-bit arches
    (PowerPC, SPARC etc) need to do explicit zero extension to meet this
    requirement, otherwise code like the following will fail.
    
      u64_value = (u64) u32_value
      ... other uses of u64_value
    
    This is because compiler could exploit the semantic described above and
    save those zero extensions for extending u32_value to u64_value, these JIT
    back-ends are expected to guarantee this through inserting extra zero
    extensions which however could be a significant increase on the code size.
    Some benchmarks show there could be ~40% sub-register writes out of total
    insns, meaning at least ~40% extra code-gen.
    
    One observation is these extra zero extensions are not always necessary.
    Take above code snippet for example, it is possible u32_value will never be
    casted into a u64, the value of high 32-bit of u32_value then could be
    ignored and extra zero extension could be eliminated.
    
    This patch implements this idea, insns defining sub-registers will be
    marked when the high 32-bit of the defined sub-register matters. For
    those unmarked insns, it is safe to eliminate high 32-bit clearnace for
    them.
    
    Algo:
     - Split read flags into READ32 and READ64.
    
     - Record index of insn that does sub-register write. Keep the index inside
       reg state and update it during verifier insn walking.
    
     - A full register read on a sub-register marks its definition insn as
       needing zero extension on dst register.
    
       A new sub-register write overrides the old one.
    
     - When propagating read64 during path pruning, also mark any insn defining
       a sub-register that is read in the pruned path as full-register.
    Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
    Signed-off-by: default avatarJiong Wang <jiong.wang@netronome.com>
    Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    5327ed3d
bpf_verifier.h 12.7 KB