Video compression question ?!
nicobi
I've got a quiet simple question... In lot's of codecs nowadays, one
is simply trying to estimate the motion between 2 frames and then one
is coding the error between those frames (thanks to dct transform, and
quantization). There are lot's of flavour of this scheme. For
instance, the new pretty brut force H264 codec (which in my point of
view is nothing but an application of all the technics known before,
and is pretty brut force indeed : multpiply the mb type, the number of
ref frames.... ) So my question is the following : would it be
possible and worth a try encoding the error beetween frames by using
wavelets on the whole reconstructed error surface ? (or may be is
there some one who already try this ???)... Is there a possibility to
mix it up with any well known codec (mpeg2, h263, h264...) ?
nicobi
nicolas
I have made a codec for windows using a wavelet transform for the encoding of
the error. The wavelets are not really an advantage and a lot of scientific work
as been done in video compression using wavelets but without significant better
results than dct.
the web site : http://www.ifrance.com/rududu
My codec is not a final version so use it only to try.
Nicolas
Thomas Richter
> So my question is the following : would it be
> possible and worth a try encoding the error beetween frames by using
> wavelets on the whole reconstructed error surface ?
It is possible and has been done, but to keep the story short: It's
typically not worth the trouble. The main problem why traditional
motion estimation and wavelets do not combine good is that the borders
of the motion blocks generate signals in the wavelet domain that are hard
to compress. With DCT, you would align the bounds of the DCT blocks with
the bounds of the motion blocks and won't have this problem.
So long,
Thomas
nicobi
> the error. The wavelets are not really an advantage and a lot of scientific work
> as been done in video compression using wavelets but without significant better
> results than dct.
>
> the web site : http://www.ifrance.com/rududu
>
> My codec is not a final version so use it only to try.
>
> Nicolas
Et sans indiscretion, est ce que tu as essayé/tenté des trucs comme :
- faire les prédictions et récolter les surfaces résiduelles.
- Encoder en wavelets 2D+t la vidéo générée par ces surfaces.
?
Je me demande si cela vaudrait le coup ??!
- Autre question, as tu essayé avec h264 ou un autre codec comme base
?
- Enfin, sais tu à quoi est du ce gain relativement faible par rapport
aux méthodes classiques ?? (hautes fréquences ??)
- appliques tu la dwt au niveau du macroblock, ou bien au niveau de
toute l'image résiduelle ??
nicobi
nicolas
understand.
> - faire les prédictions et récolter les surfaces résiduelles.
> - Encoder en wavelets 2D+t la vidéo générée par ces surfaces.
I have not tried 3D wavelet compression without motion compensation mainly
because the results are not good. There are research in using a 3D wavelet
transform + motion compensation, but the motion compensation is not performed as
you suggest but with more complex schemes. Anyway I have not tried.
> Je me demande si cela vaudrait le coup ??!
Difficult to say before trying ...
> - Autre question, as tu essayé avec h264 ou un autre codec comme base
> ?
I have not tried H264 with a wavelet transform, but I know that the improvements
that have been applied to H264 (more accurate motion compensation, long term
motion compensation) can be applied to a wavelet codec too. I have also to
improve the context based entropy coder of my codec.
> - Enfin, sais tu à quoi est du ce gain relativement faible par rapport
> aux méthodes classiques ?? (hautes fréquences ??)
The good energy compaction property of the wavelet is not as useful for the P
frames as it is for the I frames because the P frames contains a lot of high
freqs. I think the difference can be made on the coding algorithm (the zerotree
coding depends a lot on the fact that the most significant coeffs are in the low
frequency bands, and this is not true with P frames coding).
> - appliques tu la dwt au niveau du macroblock, ou bien au niveau de
> toute l'image résiduelle ??
The DWT is applied on the whole frame, so you should not be able to see the
macroblocks boundaries.
Nicolas
nicobi
> typically not worth the trouble. The main problem why traditional
> motion estimation and wavelets do not combine good is that the borders
> of the motion blocks generate signals in the wavelet domain that are hard
> to compress. With DCT, you would align the bounds of the DCT blocks with
> the bounds of the motion blocks and won't have this problem.
thanks for you answer... I understand that macroblock matching
estimation could be a bit of a trouble, even with a dct, we could
encounter troubles and annoying artifacts, without something like a
loopfilter on the encoding loop or a smooth filter at the decoder
side. Anyway, d'ya know if s simple wavelet coding on the difference
between images has been tried (on a single sequence with a good cut
detection algorithm), or even combine a classical algorithm with intra
frames wavelet coded ??
do you have any good pointers on the subject ?
nicobi
nicobi
yep ! sorry about using french in an english speaking newsgroup.
In fact you answered to all my questions, and after some readings, I
read about those complicate block motion estimation you mentionned
(mainly using an overlaped bma, or making the bma (block motion
algorithm) in the frequency domain (4x4 for instance in the HH subband
(or LL or ??!)), and alternate it with some simple wavelet on the
frame difference scheme...
But, I've got one more question : if the trouble is the high
frequencies due to the macroblock compensated edges : would it be
worth applying some kind of loopfilter one the whole frame, to smooth
up some of the High frequencies ?
Thomas Richter
> But, I've got one more question : if the trouble is the high
> frequencies due to the macroblock compensated edges : would it be
> worth applying some kind of loopfilter one the whole frame, to smooth
> up some of the High frequencies ?
Yes, sure. This has also been done. I think it would be a clever idea
to use the wavelet filter itself for this.
Greetings,
Thomas
nicolas
nicobi wrote:
> In fact you answered to all my questions, and after some readings, I
> read about those complicate block motion estimation you mentionned
> (mainly using an overlaped bma, or making the bma (block motion
> algorithm) in the frequency domain (4x4 for instance in the HH subband
> (or LL or ??!)), and alternate it with some simple wavelet on the
> frame difference scheme...
The complicated motion compensation I mentionned was mainly motion compensation
for 3D video compression. Overlapped block motion compensation is what I use in
my codec so it's not really complicated (overlapped block motion estimation is
another problem), you just need more processing. The problem with motion
compensation in the frequency domain is the fact that your maximum resolution is
half the resolution of the image (2 pxl instead of 1). and in modern compression
you need at least 1/2 pxl accuracy (1/8 in H264). Some papers tried to use the
wavelet phase using the sign as an improved accuracy, but with this you can't
have the same accuracy that you have with pxl interpolation. But as you know I'm
not an expert.
> But, I've got one more question : if the trouble is the high
> frequencies due to the macroblock compensated edges : would it be
> worth applying some kind of loopfilter one the whole frame, to smooth
> up some of the High frequencies ?
I'm using overlapped block motion compensation : there is no block edge. Try the
codec, you will see that even at high quantizer you don't see blocks. The high
frequency I'm speaking about are those of the error frame, but it's the same for
all the video codecs.
Nicolas