• Nigel Tao's avatar
    image/jpeg: reconstruct progressive images even if incomplete. · 225b223e
    Nigel Tao authored
    Fixes #14522.
    
    As I said on that issue:
    
    ----
    This is a progressive JPEG image. There are two dimensions of
    progressivity: spectral selection (variables zs and ze in scan.go,
    ranging in [0, 63]) and successive approximation (variables ah and al in
    scan.go, ranging in [0, 8), from LSB to MSB, although ah=0 implicitly
    means ah=8).
    
    For this particular image, there are three components, and the SOS
    markers contain this progression:
    
    zs, ze, ah, al:  0  0 0 0	components: 0, 1, 2
    zs, ze, ah, al:  1 63 0 0	components: 1
    zs, ze, ah, al:  1 63 0 0	components: 2
    zs, ze, ah, al:  1 63 0 2	components: 0
    zs, ze, ah, al:  1 10 2 1	components: 0
    zs, ze, ah, al: 11 63 2 1	components: 0
    zs, ze, ah, al:  1 10 1 0	components: 0
    
    The combination of all of these is complete (i.e. spectra 0 to 63 and
    bits 8 exclusive to 0) for components 1 and 2, but it is incomplete for
    component 0 (the luma component). In particular, there is no data for
    component 0, spectra 11 to 63 and bits 1 exclusive to 0.
    
    The image/jpeg code, as of Go 1.6, waits until both dimensions are
    complete before performing the de-quantization, IDCT and copy to an
    *image.YCbCr. This is the "if zigEnd != blockSize-1 || al != 0 { ...
    continue }" code and associated commentary in scan.go.
    
    Almost all progressive JPEG images end up complete in both dimensions
    for all components, but this particular image is incomplete for
    component 0, so the Go code never writes anything to the Y values of the
    resultant *image.YCbCr, which is why the broken output is so dark (but
    still looks recognizable in terms of red and blue hues).
    
    My reading of the ITU T.81 JPEG specification (Annex G) doesn't
    explicitly say that this is a valid image, but it also doesn't rule it
    out.
    
    In any case, the fix is, for progressive JPEG images, to always
    reconstruct the decoded blocks (by performing the de-quantization, IDCT
    and copy to an *image.YCbCr), regardless of whether or not they end up
    complete. Note that, in Go, the jpeg.Decode function does not return
    until the entire image is decoded, so we still only want to reconstruct
    each block once, not once per SOS (Start Of Scan) marker.
    ----
    
    A test image was also added, based on video-001.progressive.jpeg. When
    decoding that image, inserting a
    
    println("nComp, zs, ze, ah, al:", nComp, zigStart, zigEnd, ah, al)
    
    into decoder.processSOS in scan.go prints:
    
    nComp, zs, ze, ah, al: 3 0 0 0 1
    nComp, zs, ze, ah, al: 1 1 5 0 2
    nComp, zs, ze, ah, al: 1 1 63 0 1
    nComp, zs, ze, ah, al: 1 1 63 0 1
    nComp, zs, ze, ah, al: 1 6 63 0 2
    nComp, zs, ze, ah, al: 1 1 63 2 1
    nComp, zs, ze, ah, al: 3 0 0 1 0
    nComp, zs, ze, ah, al: 1 1 63 1 0
    nComp, zs, ze, ah, al: 1 1 63 1 0
    nComp, zs, ze, ah, al: 1 1 63 1 0
    
    In other words, video-001.progressive.jpeg contains 10 different scans.
    This little program below drops half of them (remembering to keep the
    "\xff\xd9" End of Image marker):
    
    ----
    package main
    
    import (
    	"bytes"
    	"io/ioutil"
    	"log"
    )
    
    func main() {
    	sos := []byte{0xff, 0xda}
    	eoi := []byte{0xff, 0xd9}
    
    	src, err := ioutil.ReadFile("video-001.progressive.jpeg")
    	if err != nil {
    		log.Fatal(err)
    	}
    	b := bytes.Split(src, sos)
    	println(len(b)) // Prints 11.
    	dst := bytes.Join(b[:5], sos)
    	dst = append(dst, eoi...)
    	if err := ioutil.WriteFile("video-001.progressive.truncated.jpeg", dst, 0666); err != nil {
    		log.Fatal(err)
    	}
    }
    ----
    
    The video-001.progressive.truncated.jpeg was converted to png via
    libjpeg and ImageMagick:
    
    djpeg -nosmooth video-001.progressive.truncated.jpeg > tmp.tga
    convert tmp.tga video-001.progressive.truncated.png
    rm tmp.tga
    
    Change-Id: I72b20cd4fb6746d36d8d4d587f891fb3bc641f84
    Reviewed-on: https://go-review.googlesource.com/21062Reviewed-by: default avatarRob Pike <r@golang.org>
    225b223e