A comparison of the effectiveness of some comparison functions in H.264.
Encoding command: x264 -q 20 -A all foreman.cif 352x288
For each non-skipped inter macroblock, the following were recorded:
- SAD of motion-compensated residual
- SSD of motion-compensated residual
- SATD of motion-compensated residual
- HCT = Sum of transformed levels, prior to quantization (but weighted based on the quant matrix)
- Bits needed to code the residual in CAVLC
- SSD of the reconstructed macroblock
- QP
- RD = reconstructed SSD + bits * exp2((QP-12)/3)
Then I plot the 4 comparison functions against RD, to see how well they work as predictors.
Also, I tried replacing SATD with HCT as the motion estimation function in x264, and the result was a gain of only .02 dB PSNR at a given bitrate.
