Test results: VP8/VP9/H.264/H.265 and new experimental AV1 codec !

[Alexey Eromenko]

Source : Futuremark 3D Mark 03 (1024x768, no AA). Scene: Mother
Nature. Captured with Fraps.

I have done both both videos as well as screenshots for easy watching.
I don't use PSNR and other maths. Just my eyes, by examining the images.
I have chosen to use low bitrate of just 512kbps on purpose, and this
is because more artifacts are visible there. If I were to choose a
higher bitrate, say 4Mbps, it would be a lot harder to find
imperfection in any of the codecs.

Points to look at: The Turtle, the butterfly, the green grass
far-and-away (top right corner).
VP8 is really the weakest of the bunch.
Plus, I have provided an uncompressed *.avi made from AV1 decoder
output (because no player today supports AV1 codec), so you can watch
for yourself !
I have taken the screenshots using VLC "frame-by-frame" feature, into
PNG, to retain quality.

Overall:

1) Good news: Background grass (top right corner) looks sharp and good
with the new AV1 codec !!! It was an Achilles' heel of VP8 codec. A
*huge* improvement there.

2) Bad news: The back of the turtle looks not very sharp with all
Google codecs VP8/VP9/AV1 at 512kb and even at 1Mbps. (I can't see
where turtle ends and where the background water begins; MPEG codecs
do better here).
Ofc. on RAW image (original) --- it is perfectly visible.

3) Text at the bottom of the images also look sharp and readable with
all the Google codecs, especially with AV1 !
It looks blurry with all MPEG codecs: AVC and HEVC. (text clarity is
useful for presentation videos, much less for Hollywood movies)

4) Another issue solved by AV1 (compared to the old VP8), is the Turtle colors:
VP8 has all colors blurred on turtle. AV1 has very good result !

5) Butterfly detail: With MPEG4-AVC it looks blocky. Not in all
scenes, mind you !
Just this one. With HEVC better, but a bit blurry. So is VP8.
AV1 provides the best image of the pack !


======
Of course one scene is nowhere near enough to decide which codec is
better, HEVC H.265 or AV1, but first results of AV1 look very
promising !

Link to the encoded videos are screenshots:
https://drive.google.com/folderview?id=0BycgkMZbeQOzdFNYNzdaNlZXY0E&usp=sharing

Thanks for your hard work,
--
-Alexey Eromenko "Technologov"

[phantom....@gmail.com]

Hello,

What encoding setting did you use?

VP8/9 does much better quality encoding when 2-pass and altref is enabled. Has it been enabled?

[Alexey Eromenko]

Yes, I used 2-pass for ALL encoders. AV1 chooses 2-pass ALWAYS.

AV1:

./aomenc -w 1024 -h 768 -v -t 2 --target-bitrate=4000 -o
/media/sf_C_DRIVE/video-test/3DMark03-f01-av1-4000k-bits.av1
/media/sf_C_DRIVE/video-test/3DMark03-f01-raw.yuv

Codec: AOMedia Project AV1 Encoder v0.1.0
Source file: /media/sf_C_DRIVE/video-test/3DMark03-f01-raw.yuv File
Type: RAW Format: I420
Destination file: /media/sf_C_DRIVE/video-test/3DMark03-f01-av1-4000k-bits.av1

Why I think it uses 2-pass?
AV1:
Pass 1/2 frame 900/901 158576B 1409b/f 42286b/s 34935 ms (25.76 fps)
Pass 2/2 frame 900/876 14453379B 10895016 ms 4.96 fpm [ETA 0:04:59]
16059F 15959F 88F 57049F 15998F 16534F 78F 60268F
16458F 16232F 15819F 89F 59479F 15714F 15167F 16474F 85F
55091F 15790Pass 2/2 frame 900/900 14937988B 132782b/f 3983463b/s
11053815 ms (0.08 fps)

See ?
AV1 reports 2-pass encoding despite the *lack* of -p 2 command parameter.
It seems to encode 2-pass anyway. Why ?

VP9:

./vpxenc -w 1024 -h 768 --codec=vp9 -p 2 -v -t 1
--target-bitrate=4000 -o 3DMark03-f01-vp9-4000k-bits.webm
3DMark03-f01-raw.yuv

Codec: WebM Project VP9 Encoder v1.5.0-1261-g6d2b79e
Encoder parameters:
g_usage = 0
g_threads = 1
g_profile = 0
g_w = 1024
g_h = 768
g_bit_depth = 8
g_input_bit_depth = 8
g_timebase.num = 1
g_timebase.den = 30
g_error_resilient = 0
g_pass = 0
g_lag_in_frames = 25
rc_dropframe_thresh = 0
rc_resize_allowed = 0
rc_scaled_width = 0
rc_scaled_height = 0
rc_resize_up_thresh = 60
rc_resize_down_thresh = 30
rc_end_usage = 0
rc_target_bitrate = 4000
rc_min_quantizer = 0
rc_max_quantizer = 63
rc_undershoot_pct = 25
rc_overshoot_pct = 25
rc_buf_sz = 6000
rc_buf_initial_sz = 4000
rc_buf_optimal_sz = 5000
rc_2pass_vbr_bias_pct = 50
rc_2pass_vbr_minsection_pct = 0
rc_2pass_vbr_maxsection_pct = 2000
kf_mode = 1
kf_min_dist = 0
kf_max_dist = 128
Pass 1/2 frame 900/901 172992B 1537b/f 46131b/s 38592 ms (23.32 fps)
Pass 2/2 frame 900/876 14457264B 8242035 ms 6.55 fpm [ETA 0:03:45]
15339F 71F 55157F 16069F 15613F 78F 53917F 15555F
15053F 78F 53704F 14429F 14617F 13675F 68F 51021F 14443F
14533F 13833FPass 2/2 frame 900/900 14908600B 132520b/f
3975626b/s 8431841 ms (0.11 fps)

As for VP8:

./vpxenc -w 1024 -h 768 --codec=vp8 -p 2 -v -t 1
--target-bitrate=4000 -o 3DMark03-f01-vp8-4000k-bits.webm
3DMark03-f01-raw.yuv

I had to use -p 2 (pass=2) manually, else it uses 1-pass, and have
double-the-size file.
i.e. VP8 with 1-pass results in 3.4 MB file, while 2-pass in 1.7 MB
file. How so ?

x264:
ffmpeg -i 3DMark03-f01.avi -vcodec libx264 -b 4000k
3DMark03-f01-h264-avc-4000k-bitrate.mp4

Also there is a HUGE speed difference.
VP8 = 12 minutes encoding
VP9 = 2:15 hrs
AV1 = 3:05 hrs !!!

System: Intel Core i7 4600U CPU + Debian Linux 7 x64.
libvpx and AV1 codecs are manually compiled (latest versions). ffmpeg
+ x264 come from the system.

And this is for a 30-second file with 30fps (900 frames total) !
Where is the mailing list for AV1 codec ? (After all it is the natural
successor of VP9/VP10 codecs).

-Alexey

[phantom....@gmail.com]

Hello,

To get faster vp9 encoding you can try this:

ffmpeg -i 3DMark03-f01.avi -c:v libvpx-vp9 -pass 1 -b:v 500K -threads 4 -speed 2 -tile-columns 2 -g 150 -aq-mode 1 -auto-alt-ref 1 -lag-in-frames 25 -an -f webm /dev/null

ffmpeg -i 3DMark03-f01.avi -c:v libvpx-vp9 -pass 2 -b:v 500K -threads 4 -speed 2 -tile-columns 2 -g 150 -aq-mode 1 -auto-alt-ref 1 -lag-in-frames 25 -c:a libopus -b:a 64k -f webm outvp9_7.webm