H.264 codec analysis

213 views
Skip to first unread message

Alex

unread,
May 12, 2011, 2:33:29 PM5/12/11
to Magic Lantern firmware development
Latest builds for 550D and 60D have the ability of changing bitrate in
CBR mode, although the meaning of the parameters isn't fully
understood yet.

550D test build:
http://groups.google.com/group/ml-devel/browse_thread/thread/850ec268bc883ceb/84f39efd77501fd2?show_docid=84f39efd77501fd2
(see my previous message in that thread for a short description of
internals)

60D test build:
http://groups.google.com/group/ml-devel/browse_thread/thread/47f8a4b186bd9a81/af85aaa3d441ea4e?show_docid=af85aaa3d441ea4e

I've also rewritten this wiki page, added a small bitrate test and
tried to remove common misconceptions about QScale.
http://magiclantern.wikia.com/wiki/Bit_rate

I start this thread for analyzing how the compression works; it may be
possible to find additional parameters, for example, subsampling mode
(currently 420).

For analysis, I will use 550D/1.0.8 firmware, for which I have the
best documentation from Arm.Indy and AJ. 5D2 204 is also a good
option.

There is a structure named AJ_Movie_CompressionRate_struct, at 0x67bc,
found also in dryos.h and named mvr_config. This is currently used for
QScale and CBR parameters. My current understanding of it is here:
https://bitbucket.org/hudson/magic-lantern/changeset/fa92f754a949

Some raw H.264 parameters (no idea about their meaning; found in
SetEncodeH264Parameter):
0xc0e10080 jp62_sizer
0xc0e100c0 jp62_seqcr1
0xc0e100d0 jp62_piccr1
0xc0e100fc jp62_miscr
0xc0e1000c jp62_opmr3
0xc0e100e0 jp62_slcr1
0xc0e100e4 jp62_slcr2

LVCDEV_H264EncodeStart(pInputAddress, pStreamAddress1, Size1,
pStreamAddress2, Size2, flag, struct(unk,unk,unk))
0x5240 MovieSize 0 1 2
0x5244 MovieMode 0 or 8, maybe crop
0x524c -1 ??
0x5260 pStreamAddress1
0x5264 image_width

0x6600: a structure with some parameters set from SetParameterH264Encode.

Is anyone familiar with the internals of H.264 and wants to help?

Simon Larcher

unread,
May 12, 2011, 3:28:40 PM5/12/11
to ml-d...@googlegroups.com
I don't know if this could be of any help for you but I just found this :
http://www.itu.int/rec/T-REC-H.264-201003-I
The PDF file available on this page seems to be pretty accurate on details.

As said on DVXUser (as SvenL) I made some tests today with the CBR settings, unfortunately my card wasn't able to hold more than 3 seconds, I think 3 seconds is still enough for my tests.
Do you want to make this thread the place where to put the tests on bitrate settings ?

2011/5/12 Alex <broscu...@gmail.com>

--
http://magiclantern.wikia.com/

To post to this group, send email to ml-d...@googlegroups.com
To unsubscribe from this group, send email to ml-devel+u...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/ml-devel?hl=en

Alex

unread,
May 12, 2011, 4:31:45 PM5/12/11
to ml-d...@googlegroups.com
Yes, you can place tests here... after all, they are helpful in seeing
how the codec works :)

The PDF is very interesting, but huge.

More findings:

- Canon's CBR algorithm seems pretty similar to my old CBRe method,
i.e. works by changing QScale. Of course, they have a much better
implementation than mine.

- QScale will not go down more than -16. This means that, on many
scenes without too much detail, bitrate can be much lower than
requested. This happens without ML, too. When this happens, it may be
dangerous, because you may have requested a bitrate higher than your
card can support, without even noticing!

In math jargon, CBR with high bitrate will converge to QScale -16.

- Reading for certain addresses around 0xc0e10000 will cause ERR80.

Simon Larcher

unread,
May 12, 2011, 7:34:53 PM5/12/11
to ml-d...@googlegroups.com
So I made my tests. I had shots of la Seine near Boulogne in bright sunlight, not much contrasty but still some shadows.
I did some grading on it, applied the same on the Firmware default quality footage and I can't see a difference because I am not comparing the same exact picture.
Then I boosted contrast and lightness up to see well and made side by side comparisons of 1600% zoom in on an uniform zone of the picture.
I am not posting my tests because I feel they are not accurate enough, it is very hard to see the difference. What I may see is that the blocks are smaller in the CBR x3 footage... but again it may very well be placebo.

I may have the occasion to make more accurate tests tomorrow at school.

2011/5/12 Alex <broscu...@gmail.com>

Simon Larcher

unread,
May 14, 2011, 8:51:18 PM5/14/11
to ml-d...@googlegroups.com
Hum... I have installed Bitrate Viewer and to load a .mov file I have to set the Load menu to All Files and it only displays bars, like a very rough visual of the bitrate variations.
I use Windows 7 64 bits.

Also, after some more tests I came to another question : what are we looking for exactly ?
It is sometimes hard to say if what we are looking at is an artifact from compression, the result of the 4:2:0 line doubling method or simply noise...

Have you got more technical infos about what this multiplier does exactly ?

2011/5/12 Simon Larcher <slarc...@gmail.com>

Alex

unread,
May 17, 2011, 7:40:32 AM5/17/11
to ml-d...@googlegroups.com
Well... if we knew what we were doing, it wouldn't be called research,
would it? [Einstein?]

A message from a user who can't post yet to mailing list:

---------- Forwarded message ----------
From: Jan Raiber <jan.r...@filmakademie.de>
Date: Tue, May 17, 2011 at 10:06 AM
Subject: H.264 codec analysis
To: broscu...@gmail.com


Hi,
some days ago i beg alex for implementing a better CBR feature. and so
he dit first steps further. the new one works much better than the
old, because at the old one, the bitrate changed to much, from under
FWdefault to beyond the speed limits of the card and/or the
card-writing-unit. And all this may happen within a single shot. In
other words it wasn't very usefull.

So why i wanted and still want to have higher bitrate? because you can
see it. In scenes with heavy detail you can see the blocking artifacts
of the compression with bare eyes. What means heavy detail? Some of
these situations, which I had the last couple of days were: Rain,
waving leaves of trees, an anthill, rought structures of walls, the
pixel-structure of a computer-monitor or high ISO-settings....
So you see, nothing very special.

I made a test where I shot with maximum ISO (6400) to see, on what
point my card will not be able to handle the datarate. a resulting
picture, you can see here:
surrandom.com/toDownload/CBR.jpg
(it's two times the upper right part of the picture at 100% size)

Here it's very easy to see the difference between FWdefault (around
45mbit) and CBR 1,7 (arround 65mbit) You can recognize, that the
default setting will produce a very blurry image with much blocking,
like a bad youtube-video. this is nothing you want to have in
postproduction.

On the right picture you see much more detail, in this case the
original grain of the image sensor. (which is nothing you wanna have
as well, but hopefully you will never be in situation to shoot with
6400 ISO ...)
So detailed grain is better, it looks much more natural, but don't
misunderstand, it's not the case, that you have more of this grain on
the right image, in the left you have the grain plus the degrading,
plus the blocking, which is not easy to handle in color correction.

But what kind of color correction do i mean, you can do heavy things
like much contrast or desaturating and you will not see anything more.
But if you want to brighten the picture, or increase the color, you
will come in a mess of blocks.
And the worse thing you can do with heavily compressed material is
selective color-correction, where you pic a color (blue of sky or
green of gras) and change it separately from the other colors.
Try it with your material and you will see, what I mean. With
uncompressed material from a telecine f.e. you can turn one yellow car
out of a traffic jam into a blue one, without anybody will notice it.
with compressed material you must be the master of masks to isolate
it.

But there is an other very good way to see, how much reduced is your
material. put it in an application like after effects and have a look
at the channel switch beneath the mainframe.
click it to show only the red, or the blue channel. switch to full
quality and 100% view. Now choose a part of the picture with detail
and you will see what compression is doing to you picture. The eye is
less sensitive to red and blue (instead of green or lightness) so the
reduction is more intense here.

And what do you see? Heaps of blocks all over the frame. This has
nothing to do with 4:2:0 subsampling (this should only look twice as
pixie than normal) and the only way to reduce it, is cranking up the
data-rate as much as possible. You see this blocks even is scenes with
less detail, less of them, but you see them, because the rate goes
down to hold same image quality.

If you think the best thing for grading and post would be having 4:2:2
subsampling than you'r right and wrong. it's only better, if you can
raise the data-rate, because, you got a quarter more pixeldata to
compress and to safe. So remember the picture above: is it useful to
have a picture with more detail, which has to be more compressed? i
don't think so.
So the first step to improve image quality with this cams is the
highest possible data-rate, if it works (a CBR on a hard limit), than
we can think of a better subsampling or maybe a better bitdepth (10
instead of 8 bit is quarter more data too) which would give us more
flexibility in lightness and contrast than the subsampling. 1024 steps
in lightness instead of 256 (!) in my eyes, it's a better deal.

But first: less compression.
Maybe there are other possibilities with the codec because i had the
experience with AVC-HD from Sony with less rate (35mbit) but better
image quality (in terms of compression) than the canon codec (which is
h264 too). But maybe it's limited by the processing power of the image
processor. But i'm not in this terms to know, if it has something to
do with a better use of B-frames....???

So long, have a nice day,
greets and cheers,
Jan

jraiber

unread,
May 13, 2011, 3:27:35 AM5/13/11
to Magic Lantern firmware development
Hi I dit some test as well.
you need to look close to the red or blue channel of the material you
shot, because they are much more compressed than green or lightness
(which you recognize most if you watch a normal rgb picture)
I dit a test with very high ISO because its the hardest thing for
compression, its a lot of detail and so you can compare the most
complicate-to-compress image. maybe a tree with waving leaves gives
same results or a fine pattern, or an anthill...

if you shoot a picture with not much movement and detail you will get
same results with fw-default. in former builds it was the case that
bitrate changed too much. so complex pictures overload the card and
soft pictures made a rate less than firmware, it was always a
compromise and you had to adjust your setting to the scene, you shot.
not very handy for documentary shooting.

Now, so is my opinion after first tests, we got a step further. yes of
course, its' possible to tweak it a little bit and 422 subsampling
will go a next step, but i'm happy already. :)

Yeah!

so my pic to compare high ISO recording to simulate much detail you
can see here with bare eyes already in the rgb channels. FW-default is
much softer, more blocky, looks like a youtube video. the 1.7x (which
my card can hold as long as i wish) lets you see much more of the
original sensor-highISO-grain. And the same thing will happen, if you
shoot a very very complex scene.

http://www.surrandom.com/toDOWNLOAD/CBR.jpg

never the less a reallife shooting test will show, that you are likely
to go up to 2x or more because you will not often have so much detail
like ISO 6400. And thats the point, where it can be improved. A kind
of hard limit at a specific datarate, which corresponds to card-speed
might be the goal.

so long happy shooting, and thanks to Alex,
Jan


On 13 Mai, 01:34, Simon Larcher <slarche...@gmail.com> wrote:
> So I made my tests. I had shots of la Seine near Boulogne in bright
> sunlight, not much contrasty but still some shadows.
> I did some grading on it, applied the same on the Firmware default quality
> footage and I can't see a difference because I am not comparing the same
> exact picture.
> Then I boosted contrast and lightness up to see well and made side by side
> comparisons of 1600% zoom in on an uniform zone of the picture.
> I am not posting my tests because I feel they are not accurate enough, it is
> very hard to see the difference. What I may see is that the blocks are
> smaller in the CBR x3 footage... but again it may very well be placebo.
>
> I may have the occasion to make more accurate tests tomorrow at school.
>
> 2011/5/12 Alex <broscutama...@gmail.com>
>
>
>
> > Yes, you can place tests here... after all, they are helpful in seeing
> > how the codec works :)
>
> > The PDF is very interesting, but huge.
>
> > More findings:
>
> > - Canon's CBR algorithm seems pretty similar to my old CBRe method,
> > i.e. works by changing QScale. Of course, they have a much better
> > implementation than mine.
>
> > - QScale will not go down more than -16. This means that, on many
> > scenes without too much detail, bitrate can be much lower than
> > requested. This happens without ML, too. When this happens, it may be
> > dangerous, because you may have requested a bitrate higher than your
> > card can support, without even noticing!
>
> > In math jargon, CBR with high bitrate will converge to QScale -16.
>
> > - Reading for certain addresses around 0xc0e10000 will cause ERR80.
>
> > On Thu, May 12, 2011 at 9:28 PM, Simon Larcher <slarche...@gmail.com>
> > wrote:
> > > I don't know if this could be of any help for you but I just found this :
> > >http://www.itu.int/rec/T-REC-H.264-201003-I
> > > The PDF file available on this page seems to be pretty accurate on
> > details.
>
> > > As said on DVXUser (as SvenL) I made some tests today with the CBR
> > settings,
> > > unfortunately my card wasn't able to hold more than 3 seconds, I think 3
> > > seconds is still enough for my tests.
> > > Do you want to make this thread the place where to put the tests on
> > bitrate
> > > settings ?
>
> > > 2011/5/12 Alex <broscutama...@gmail.com>
>
> > >> Latest builds for 550D and 60D have the ability of changing bitrate in
> > >> CBR mode, although the meaning of the parameters isn't fully
> > >> understood yet.
>
> > >> 550D test build:
>
> >http://groups.google.com/group/ml-devel/browse_thread/thread/850ec268...
> > >> (see my previous message in that thread for a short description of
> > >> internals)
>
> > >> 60D test build:
>
> >http://groups.google.com/group/ml-devel/browse_thread/thread/47f8a4b1...

jraiber

unread,
May 16, 2011, 5:13:53 PM5/16/11
to Magic Lantern firmware development
Hi,
some days ago i beg alex for implementing a better CBR feature. and so
he dit first steps further. the new one works much better than the
old, because at the old one, the bitrate changed to much, from under
FWdefault to beyond the speed limits of the card and/or the card-
writing-unit. And all this may happen within a single shot. In other
words it wasn't very usefull.

So why i wanted and still want to have higher bitrate? because you can
see it. In scenes with heavy detail you can see the blocking artifacts
of the compression with bare eyes. What means heavy detail? Some of
these situations, which I had the last couple of days were: Rain,
waving leaves of trees, an anthill, rought structures of walls, the
pixel-structure of a computer-monitor or high ISO-settings....
So you see, nothing very special.

I made a test where I shot with maximum ISO (6400) to see, on what
point my card will not be able to handle the datarate. a resulting
picture, you can see here:
surrandom.com/toDownload/CBR.jpg
(it's two times the upper right portion of the picture at 100% size)
do with a better use of B Frames....???

So long, have a nice day,
greets and cheers,
Jan




On 15 Mai, 02:51, Simon Larcher <slarche...@gmail.com> wrote:
> Hum... I have installed Bitrate Viewer and to load a .mov file I have to set
> the Load menu to All Files and it only displays bars, like a very rough
> visual of the bitrate variations.
> I use Windows 7 64 bits.
>
> Also, after some more tests I came to another question : what are we looking
> for exactly ?
> It is sometimes hard to say if what we are looking at is an artifact from
> compression, the result of the 4:2:0 line doubling method or simply noise...
>
> Have you got more technical infos about what this multiplier does exactly ?
>
> 2011/5/12 Simon Larcher <slarche...@gmail.com>
>
>
>
> > So I made my tests. I had shots of la Seine near Boulogne in bright
> > sunlight, not much contrasty but still some shadows.
> > I did some grading on it, applied the same on the Firmware default quality
> > footage and I can't see a difference because I am not comparing the same
> > exact picture.
> > Then I boosted contrast and lightness up to see well and made side by side
> > comparisons of 1600% zoom in on an uniform zone of the picture.
> > I am not posting my tests because I feel they are not accurate enough, it
> > is very hard to see the difference. What I may see is that the blocks are
> > smaller in the CBR x3 footage... but again it may very well be placebo.
>
> > I may have the occasion to make more accurate tests tomorrow at school.
>
> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> Yes, you can place tests here... after all, they are helpful in seeing
> >> how the codec works :)
>
> >> The PDF is very interesting, but huge.
>
> >> More findings:
>
> >> - Canon's CBR algorithm seems pretty similar to my old CBRe method,
> >> i.e. works by changing QScale. Of course, they have a much better
> >> implementation than mine.
>
> >> - QScale will not go down more than -16. This means that, on many
> >> scenes without too much detail, bitrate can be much lower than
> >> requested. This happens without ML, too. When this happens, it may be
> >> dangerous, because you may have requested a bitrate higher than your
> >> card can support, without even noticing!
>
> >> In math jargon, CBR with high bitrate will converge to QScale -16.
>
> >> - Reading for certain addresses around 0xc0e10000 will cause ERR80.
>
> >> On Thu, May 12, 2011 at 9:28 PM, Simon Larcher <slarche...@gmail.com>
> >> wrote:
> >> > I don't know if this could be of any help for you but I just found this
> >> :
> >> >http://www.itu.int/rec/T-REC-H.264-201003-I
> >> > The PDF file available on this page seems to be pretty accurate on
> >> details.
>
> >> > As said on DVXUser (as SvenL) I made some tests today with the CBR
> >> settings,
> >> > unfortunately my card wasn't able to hold more than 3 seconds, I think 3
> >> > seconds is still enough for my tests.
> >> > Do you want to make this thread the place where to put the tests on
> >> bitrate
> >> > settings ?
>
> >> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> >> Latest builds for 550D and 60D have the ability of changing bitrate in
> >> >> CBR mode, although the meaning of the parameters isn't fully
> >> >> understood yet.
>
> >> >> 550D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/850ec268...
> >> >> (see my previous message in that thread for a short description of
> >> >> internals)
>
> >> >> 60D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/47f8a4b1...

jraiber

unread,
May 16, 2011, 5:14:35 PM5/16/11
to Magic Lantern firmware development
> Hum... I have installed Bitrate Viewer and to load a .mov file I have to set
> the Load menu to All Files and it only displays bars, like a very rough
> visual of the bitrate variations.
> I use Windows 7 64 bits.
>
> Also, after some more tests I came to another question : what are we looking
> for exactly ?
> It is sometimes hard to say if what we are looking at is an artifact from
> compression, the result of the 4:2:0 line doubling method or simply noise...
>
> Have you got more technical infos about what this multiplier does exactly ?
>
> 2011/5/12 Simon Larcher <slarche...@gmail.com>
>
>
>
> > So I made my tests. I had shots of la Seine near Boulogne in bright
> > sunlight, not much contrasty but still some shadows.
> > I did some grading on it, applied the same on the Firmware default quality
> > footage and I can't see a difference because I am not comparing the same
> > exact picture.
> > Then I boosted contrast and lightness up to see well and made side by side
> > comparisons of 1600% zoom in on an uniform zone of the picture.
> > I am not posting my tests because I feel they are not accurate enough, it
> > is very hard to see the difference. What I may see is that the blocks are
> > smaller in the CBR x3 footage... but again it may very well be placebo.
>
> > I may have the occasion to make more accurate tests tomorrow at school.
>
> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> Yes, you can place tests here... after all, they are helpful in seeing
> >> how the codec works :)
>
> >> The PDF is very interesting, but huge.
>
> >> More findings:
>
> >> - Canon's CBR algorithm seems pretty similar to my old CBRe method,
> >> i.e. works by changing QScale. Of course, they have a much better
> >> implementation than mine.
>
> >> - QScale will not go down more than -16. This means that, on many
> >> scenes without too much detail, bitrate can be much lower than
> >> requested. This happens without ML, too. When this happens, it may be
> >> dangerous, because you may have requested a bitrate higher than your
> >> card can support, without even noticing!
>
> >> In math jargon, CBR with high bitrate will converge to QScale -16.
>
> >> - Reading for certain addresses around 0xc0e10000 will cause ERR80.
>
> >> On Thu, May 12, 2011 at 9:28 PM, Simon Larcher <slarche...@gmail.com>
> >> wrote:
> >> > I don't know if this could be of any help for you but I just found this
> >> :
> >> >http://www.itu.int/rec/T-REC-H.264-201003-I
> >> > The PDF file available on this page seems to be pretty accurate on
> >> details.
>
> >> > As said on DVXUser (as SvenL) I made some tests today with the CBR
> >> settings,
> >> > unfortunately my card wasn't able to hold more than 3 seconds, I think 3
> >> > seconds is still enough for my tests.
> >> > Do you want to make this thread the place where to put the tests on
> >> bitrate
> >> > settings ?
>
> >> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> >> Latest builds for 550D and 60D have the ability of changing bitrate in
> >> >> CBR mode, although the meaning of the parameters isn't fully
> >> >> understood yet.
>
> >> >> 550D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/850ec268...
> >> >> (see my previous message in that thread for a short description of
> >> >> internals)
>
> >> >> 60D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/47f8a4b1...

jraiber

unread,
May 16, 2011, 5:21:58 PM5/16/11
to Magic Lantern firmware development
> Hum... I have installed Bitrate Viewer and to load a .mov file I have to set
> the Load menu to All Files and it only displays bars, like a very rough
> visual of the bitrate variations.
> I use Windows 7 64 bits.
>
> Also, after some more tests I came to another question : what are we looking
> for exactly ?
> It is sometimes hard to say if what we are looking at is an artifact from
> compression, the result of the 4:2:0 line doubling method or simply noise...
>
> Have you got more technical infos about what this multiplier does exactly ?
>
> 2011/5/12 Simon Larcher <slarche...@gmail.com>
>
>
>
> > So I made my tests. I had shots of la Seine near Boulogne in bright
> > sunlight, not much contrasty but still some shadows.
> > I did some grading on it, applied the same on the Firmware default quality
> > footage and I can't see a difference because I am not comparing the same
> > exact picture.
> > Then I boosted contrast and lightness up to see well and made side by side
> > comparisons of 1600% zoom in on an uniform zone of the picture.
> > I am not posting my tests because I feel they are not accurate enough, it
> > is very hard to see the difference. What I may see is that the blocks are
> > smaller in the CBR x3 footage... but again it may very well be placebo.
>
> > I may have the occasion to make more accurate tests tomorrow at school.
>
> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> Yes, you can place tests here... after all, they are helpful in seeing
> >> how the codec works :)
>
> >> The PDF is very interesting, but huge.
>
> >> More findings:
>
> >> - Canon's CBR algorithm seems pretty similar to my old CBRe method,
> >> i.e. works by changing QScale. Of course, they have a much better
> >> implementation than mine.
>
> >> - QScale will not go down more than -16. This means that, on many
> >> scenes without too much detail, bitrate can be much lower than
> >> requested. This happens without ML, too. When this happens, it may be
> >> dangerous, because you may have requested a bitrate higher than your
> >> card can support, without even noticing!
>
> >> In math jargon, CBR with high bitrate will converge to QScale -16.
>
> >> - Reading for certain addresses around 0xc0e10000 will cause ERR80.
>
> >> On Thu, May 12, 2011 at 9:28 PM, Simon Larcher <slarche...@gmail.com>
> >> wrote:
> >> > I don't know if this could be of any help for you but I just found this
> >> :
> >> >http://www.itu.int/rec/T-REC-H.264-201003-I
> >> > The PDF file available on this page seems to be pretty accurate on
> >> details.
>
> >> > As said on DVXUser (as SvenL) I made some tests today with the CBR
> >> settings,
> >> > unfortunately my card wasn't able to hold more than 3 seconds, I think 3
> >> > seconds is still enough for my tests.
> >> > Do you want to make this thread the place where to put the tests on
> >> bitrate
> >> > settings ?
>
> >> > 2011/5/12 Alex <broscutama...@gmail.com>
>
> >> >> Latest builds for 550D and 60D have the ability of changing bitrate in
> >> >> CBR mode, although the meaning of the parameters isn't fully
> >> >> understood yet.
>
> >> >> 550D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/850ec268...
> >> >> (see my previous message in that thread for a short description of
> >> >> internals)
>
> >> >> 60D test build:
>
> >>http://groups.google.com/group/ml-devel/browse_thread/thread/47f8a4b1...

jraiber

unread,
May 23, 2011, 6:03:48 PM5/23/11
to Magic Lantern firmware development
Ok, now it seems, that my posts come up. Yes i tried it several times,
because i wrote a lot....

today i was thinking, why does the buffer of the cam (60D) do over-
floating so fast with higher bitrates (80Mbit/s). this buffer must be
very tiny in comparison to the buffer the cam has for the photo burst.
It can take over 4 (Canon says 5.3) RAW pictures a second which is
about 100MByte ...

Maybe we can address this buffer for video too???

Sidmar Holloman

unread,
May 23, 2011, 6:38:42 PM5/23/11
to ml-d...@googlegroups.com
so when you increase the bitrate, you basically increase the resolution? what would you say the bitrate of 2k, 3k ,4k, etc be?

--- On Fri, 5/13/11, jraiber <jra...@web.de> wrote:

> > >> For more options, visit this group at
> > >>http://groups.google.com/group/ml-devel?hl=en
>
> > > --
> > >http://magiclantern.wikia.com/
>
> > > To post to this group, send email to ml-d...@googlegroups.com
> > > To unsubscribe from this group, send email to

> > > For more options, visit this group at
> > >http://groups.google.com/group/ml-devel?hl=en
>
> > --
> >http://magiclantern.wikia.com/
>
> > To post to this group, send email to ml-d...@googlegroups.com
> > To unsubscribe from this group, send email to

> > For more options, visit this group at
> >http://groups.google.com/group/ml-devel?hl=en

--
http://magiclantern.wikia.com/

To post to this group, send email to ml-d...@googlegroups.com
To unsubscribe from this group, send email to ml-devel+unsub...@googlegroups.com

Alex

unread,
May 23, 2011, 6:41:39 PM5/23/11
to ml-d...@googlegroups.com
http://en.wikipedia.org/wiki/Bit_rate

This is not a place for learning the very basics of using a camera.

To unsubscribe from this group, send email to ml-devel+u...@googlegroups.com

Alex

unread,
May 25, 2011, 8:38:30 AM5/25/11
to ml-d...@googlegroups.com
More findings:

There are combinations of CBR tuning parameters (OptSize and GopOptSize) for each resolution and frame rate. AJ noticed a similar pattern in 5D2.

Functions used by ML (eg. mvrSetFullHDOptSize ) will set the same values for all frame rates. This confused me and made me think the parameters depend only on the resolution (which is not true, and resulted in subtle bugs).

Due to this, it's not possible to revert to default compression setings using calls to mvrSet***. It may be possible by rewriting the entire mvr_config structure, but I'd like to avoid that (i.e. effects in case of incorrect stubs may be bad).

If this theory is right, the attached autoexec.bin (only for 550D/1.0.9) should fix CBR settings for all possible resolution and frame rates.

Also, this autoexec.bin will ask for a restart whenever the "FW default" setting may be different than the true default (i.e. without ML). When ML shows in the "FW default" without asking for restart, it means the mvr_config structure is untouched.

Things to test (in all combinations of resolution and frame rate):

- CBR 1x should be identical to FW default
- CBR 2x should have bitrate twice as big (i.e. around 90mbps in 1080p).
- CBR 0.5x should have bitrate twice as small.
- CBR 0.1x should have bitrate twice 10x smaller.
- Bitrate graph should be as smooth as in FW default setting.

Please note QScale won't go outside -16...+16 range. In this case, bitrate will vary in CBR mode, and this is normal.

Changeset: https://bitbucket.org/hudson/magic-lantern/changeset/e40b170cd81e
autoexec.bin

Manolis Nikiforakis

unread,
May 31, 2011, 7:45:06 PM5/31/11
to Magic Lantern firmware development
I hope I am no spamming the group, as I am a member for only a few
minutes now :-) but I was playing with CBR just today, so I can
confirm the following:
CBR 0.5x is indeed 50% smaller that CBR 1
CBR 1x is same as FW default
QScale -4 in two tests gave me same file size with CBR 0.5x (but in
another test I had filesize of QS -4 = to CBR 1x)

My tests are with 550D on 720p50, with may's 25 ML version. I did not
look at bit rate, only file size. There was no msg asking to reboot.

I dont know if my findings are of any interest as they seem obvious.
Personally I decided to use CBR 0.5x for "average quality long
duration" footage.

Manolis
>  autoexec.bin
> 232KViewDownload

jraiber

unread,
Jun 2, 2011, 3:04:56 PM6/2/11
to Magic Lantern firmware development
Hi,
i'd like to do all those testings, but i've a 60d. So the question: is
this autoexec.bin the same like alpha12 for 60D?
If not my test would be worthless.
(btw. bitrates with alpha12 seems to be much smoother than before, so
less buffer overflows, but in tripot-shots a recognized heavy
difference between i-frames and the others. I-frames has twice the
data of B or P, but i have no comparison with FW-defaults, should make
some tests)

jan


>
> Things to test (in all combinations of resolution and frame rate):
>
> - CBR 1x should be identical to FW default
> - CBR 2x should have bitrate twice as big (i.e. around 90mbps in 1080p).
> - CBR 0.5x should have bitrate twice as small.
> - CBR 0.1x should have bitrate twice 10x smaller.
> - Bitrate graph should be as smooth as in FW default setting.
>
> Please note QScale won't go outside -16...+16 range. In this case, bitrate
> will vary in CBR mode, and this is normal.
>
> Changeset:https://bitbucket.org/hudson/magic-lantern/changeset/e40b170cd81e
>
>  autoexec.bin
> 232KAnzeigenHerunterladen

Alex

unread,
Jun 2, 2011, 6:39:05 PM6/2/11
to Magic Lantern firmware development
You can use alpha12, it has the same implementation of CBR.

robli...@gmail.com

unread,
Jun 2, 2011, 7:58:40 PM6/2/11
to ml-d...@googlegroups.com
Yy
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: jraiber <jra...@web.de>
Sender: ml-d...@googlegroups.com
Date: Thu, 2 Jun 2011 12:04:56
To: Magic Lantern firmware development<ml-d...@googlegroups.com>
Reply-To: ml-d...@googlegroups.com
Subject: [ML] Re: H.264 codec analysis

jraiber

unread,
Jun 15, 2011, 5:58:35 PM6/15/11
to Magic Lantern firmware development
Hi folks, hi Alex,
with a little delay i made the tests.

you can download the full pdf with all graphs here:
www.ml.surrandom.com/bitrates.pdf

- I dit several bitrat-tests with different framerates
- I’ve an european 60D, so i’m not able to test 30FPS
- Firmware Ver. 1.0.9 with ML for 60D alpha12 (june2011)
- FW default was made with a clean card without ML (last column is for
comparison with FWdefault set by ML-menu)
- average bitrates mesured with quicktime-player (filesize/sec) (but
might include audio)
- peaks and images made with autodesk cleaner
- blue bars are i-frames, which generates the peaks
- i shot an out of focus picture, which come to focus (to generate a
test with different detail)
- i shot with ISO 640
- as an extra for least bitrate i shot a black picture (lenscap) with
ISO 100 with 25 and 50 FPS
- PictureStyle: neutral
- HTP: off
- ALO: off
- fresh low-level formated (fat 32 ) SanDisk Extreme 30MB/s SDHC card
- buffer overflow at CBRx 2.0 after few seconds of focus


so long, hopefully it's still interesting after the big hype about
FULL-HD HDMI-out and external recording with a cheap device.
Jan

jraiber

unread,
Jun 16, 2011, 2:11:22 AM6/16/11
to Magic Lantern firmware development
yes yes, knowing the basics of the cam is very important...
have to switch to ntsc first, than i can use 30 and 60 FpS too.
maybe i'll add this tonight.

cheers,
jan

jraiber

unread,
Jun 23, 2011, 4:33:40 AM6/23/11
to Magic Lantern firmware development
hi had no time the last days to make the additional 30 and 60 fps
tests but a friend had sent me an interesting comparison between
420/422 and 8bit/10bit for C/h264 codecs in terms of quality
improvements and datarate increasings. before i read this i thought
that a switch to 422 would increase the need for a much higher rate,
but it doesn't....

have a look at it: http://broadcastengineering.com/hdtv/avch-encoding/index.html

greets,
jan
Reply all
Reply to author
Forward
0 new messages