• Nigel Tao's avatar
    exp/html: make the tokenizer return atoms for tag tokens. · cd21eff7
    Nigel Tao authored
    This is part 1 of a 2 part changelist. Part 2 contains the mechanical
    change to parse.go to compare atoms (ints) instead of strings.
    
    The overall effect of the two changes are:
    benchmark                      old ns/op    new ns/op    delta
    BenchmarkParser                  4462274      4058254   -9.05%
    BenchmarkRawLevelTokenizer        913202       912917   -0.03%
    BenchmarkLowLevelTokenizer       1268626      1267836   -0.06%
    BenchmarkHighLevelTokenizer      1947305      1968944   +1.11%
    
    R=rsc
    CC=andybalholm, golang-dev, r
    https://golang.org/cl/6305053
    cd21eff7
token.go 19.5 KB