experimental branch

198 views
Skip to first unread message

James Zern

unread,
Nov 17, 2011, 5:41:06 PM11/17/11
to WebP Discussion
Hi,
I just pushed a new experimental branch to our git repository [1].
This adds code for webp with alpha (via the new mux interface [2]) as
well as a new lossless encoder.

As the name suggests, the code is a work in progress and is suitable
only for testing at this point. If you update an existing clone of the
repository or setup a new one [3], you can checkout the code with:
$ git checkout -b experimental origin/experimental

The easiest way to build right now is with autoconf:
$ ./autogen.sh && ./configure --enable-experimental && make
Note this requires zlib to build.

Binaries and a more detailed news post are coming soon...stay tuned!

[1]: http://git.chromium.org/gitweb/?p=webm/libwebp.git;a=shortlog;h=refs/heads/experimental
[2]: https://groups.google.com/a/webmproject.org/group/webp-discuss/browse_thread/thread/4ab76cbde89e6ade#
[3]: http://www.webmproject.org/code/#libwebp_webp_image_library

James Zern

unread,
Nov 17, 2011, 10:05:21 PM11/17/11
to WebP Discussion
On Nov 17, 2:41 pm, James Zern <jz...@google.com> wrote:
> Hi,
> I just pushed a new experimental branch to our git repository [1].
> This adds code for webp with alpha (via the new mux interface [2]) as
> well as a new lossless encoder.
>
> As the name suggests, the code is a work in progress and is suitable
> only for testing at this point. If you update an existing clone of the
> repository or setup a new one [3], you can checkout the code with:
>  $ git checkout -b experimental origin/experimental
>
> The easiest way to build right now is with autoconf:
>  $ ./autogen.sh && ./configure --enable-experimental && make
> Note this requires zlib to build.
>
> Binaries and a more detailed news post are coming soon...stay tuned!
>
> [1]:http://git.chromium.org/gitweb/?p=webm/libwebp.git;a=shortlog;h=refs/...
> [2]:https://groups.google.com/a/webmproject.org/group/webp-discuss/browse...
> [3]:http://www.webmproject.org/code/#libwebp_webp_image_library

Added a few binary builds [4][5][6]. The lossless code needs a little
work to compile on Windows so those will come in a bit.

[4]: http://webp.googlecode.com/files/libwebp-experimental-linux-x86-32.tar.gz
[5]: http://webp.googlecode.com/files/libwebp-experimental-linux-x86-64.tar.gz
[6]: http://webp.googlecode.com/files/libwebp-experimental-mac-10.6.tar.gz

James Zern

unread,
Nov 17, 2011, 11:59:40 PM11/17/11
to WebP Discussion

On Nov 17, 7:05 pm, James Zern <jz...@google.com> wrote:
> On Nov 17, 2:41 pm, James Zern <jz...@google.com> wrote:
>
> > Hi,
> > I just pushed a new experimental branch to our git repository [1].
> > This adds code for webp with alpha (via the new mux interface [2]) as
> > well as a new lossless encoder.
>
> > As the name suggests, the code is a work in progress and is suitable
> > only for testing at this point. If you update an existing clone of the
> > repository or setup a new one [3], you can checkout the code with:
> >  $ git checkout -b experimental origin/experimental
>
> > The easiest way to build right now is with autoconf:
> >  $ ./autogen.sh && ./configure --enable-experimental && make
> > Note this requires zlib to build.
>
> > Binaries and a more detailed news post are coming soon...stay tuned!
>
> > [1]:http://git.chromium.org/gitweb/?p=webm/libwebp.git;a=shortlog;h=refs/...
> > [2]:https://groups.google.com/a/webmproject.org/group/webp-discuss/browse...
> > [3]:http://www.webmproject.org/code/#libwebp_webp_image_library
>
> Added a few binary builds [4][5][6]. The lossless code needs a little
> work to compile on Windows so those will come in a bit.
>

> [4]:http://webp.googlecode.com/files/libwebp-experimental-linux-x86-32.ta...
> [5]:http://webp.googlecode.com/files/libwebp-experimental-linux-x86-64.ta...
> [6]:http://webp.googlecode.com/files/libwebp-experimental-mac-10.6.tar.gz

We'll go with a mingw build for windows for the time being [7].
You will need a bash shell to run the included script -- no batch
file...sorry ---, but the binaries will run under the builtin command
prompt.

[7]: http://webp.googlecode.com/files/libwebp-experimental-mingw32.tar.gz

mark

unread,
Nov 18, 2011, 9:55:50 AM11/18/11
to WebP Discussion
Using the binaries provided, png2webpll hangs with CPU @ ~100% on any
png I try. OSX 10.7.2. Here's some I tried (searched for transparent
png):

http://upload.wikimedia.org/wikipedia/commons/e/e5/Post-it-note-transparent.png
http://fc01.deviantart.net/fs71/f/2010/048/a/7/Crystal_Ball_Transparent_png_by_manoluv.png
http://www.coolwebstock.com/upimg/allimg/071106/200712721572423477802.png

On Nov 17, 11:05 pm, James Zern <jz...@google.com> wrote:
> On Nov 17, 2:41 pm, James Zern <jz...@google.com> wrote:
>
>
>
>
>
>
>
>
>
> > Hi,
> > I just pushed a new experimental branch to our git repository [1].
> > This adds code for webp with alpha (via the new mux interface [2]) as
> > well as a new lossless encoder.
>
> > As the name suggests, the code is a work in progress and is suitable
> > only for testing at this point. If you update an existing clone of the
> > repository or setup a new one [3], you can checkout the code with:
> >  $ git checkout -b experimental origin/experimental
>
> > The easiest way to build right now is with autoconf:
> >  $ ./autogen.sh && ./configure --enable-experimental && make
> > Note this requires zlib to build.
>
> > Binaries and a more detailed news post are coming soon...stay tuned!
>
> > [1]:http://git.chromium.org/gitweb/?p=webm/libwebp.git;a=shortlog;h=refs/...
> > [2]:https://groups.google.com/a/webmproject.org/group/webp-discuss/browse...
> > [3]:http://www.webmproject.org/code/#libwebp_webp_image_library
>
> Added a few binary builds [4][5][6]. The lossless code needs a little
> work to compile on Windows so those will come in a bit.
>

Szabolcs Péter

unread,
Nov 18, 2011, 1:09:12 PM11/18/11
to webp-d...@webmproject.org
Hello,
can you comment on what algorithm it uses for lossless? How come it's so much better then PNG's zlib?

Thanks,
 syp

James Zern

unread,
Nov 18, 2011, 1:54:47 PM11/18/11
to WebP Discussion

On Nov 18, 10:09 am, Szabolcs Péter <syp...@gmail.com> wrote:
> Hello,
> can you comment on what algorithm it uses for lossless? How come it's so
> much better then PNG's zlib?
>

The current README in src/lossless gives brief overview; pasting
inline for convenience:
"""
This directory contains a lossless compression algorithm that
compresses partially translucent images typically 30 % more densely
than pngcrush. The methods used include:

Backward references (with 2d locality for the closest 120 pixel
positions).

Prefixed Huffman coding, like flate, but squares of 2^n x 2^n pixels
can have their own Huffman code. Further, green, red, blue and alpha
have their separate entropy codes. However, green is special: green is
encoded with copy-lengths and "palette hash codes".

Spatial color prediction (for 2^n x 2^n pixel squares), slightly
similar to
Paeth predictor in png.

Palette hash codes -- multiplicative hashing based "emerging palette"
compression.

Intercomponent color prediction (for 2^n x 2^n pixel squares).

Sub-resolution images of the entropy codes, spatial color prediction,
and intercomponent color prediction are encoded recursively with the
image format itself.
"""

Alex Ratushnyak

unread,
Nov 18, 2011, 2:08:24 PM11/18/11
to WebP Discussion, mark
"Using the binaries provided, png2webpll hangs with CPU @ ~100% on any
png I try."

Mark, I also thought it hangs, but then I tried it with -c 0:
> png2webpll image.png -o image.webpll -c 0
and understood that it doesn't hang,
just works slowly after printing nothing.

James,
should I add this version to LPCB ( http://imagecompression.info/gralic/LPCB.html
)
or wait for a better one?

Alex Ratushnyak

unread,
Nov 18, 2011, 2:13:35 PM11/18/11
to WebP Discussion
Mark,
it doesn't hang, try it with "-c 0" at the end of command line, and it
will run faster.

James,
should I add current version to LPCB ( http://imagecompression.info/gralic/LPCB.html

James Zern

unread,
Nov 18, 2011, 2:45:43 PM11/18/11
to WebP Discussion

On Nov 18, 6:55 am, mark <mark.nord...@gmail.com> wrote:
> Using the binaries provided, png2webpll hangs with CPU @ ~100% on any
> png I try. OSX 10.7.2. Here's some I tried (searched for transparent
> png):
>

> http://upload.wikimedia.org/wikipedia/commons/e/e5/Post-it-note-trans...http://fc01.deviantart.net/fs71/f/2010/048/a/7/Crystal_Ball_Transpare...http://www.coolwebstock.com/upimg/allimg/071106/200712721572423477802...
>

It could be an issue on 10.7, but note the encoder is currently a bit
slow...from 10.6.8 for the mentioned images:
$ for f in *.png; do time ./png2webpll $f -o $f.webpll; done
png = 80043, webpll = 36500 bytes (0.456005)
real 0m18.886s
user 0m18.832s
sys 0m0.045s

png = 1262604, webpll = 153312 bytes (0.121425)
real 2m6.861s
user 2m6.629s
sys 0m0.220s

png = 360820, webpll = 231841 bytes (0.642539)
real 1m58.823s
user 1m58.546s
sys 0m0.257s

Alex Ratushnyak

unread,
Nov 18, 2011, 6:13:50 PM11/18/11
to WebP Discussion
> note the encoder is currently a bit slow

Furthermore, it's a bit crashing sometimes:

C:\...ibwebp-experimental-mingw32\libwebp-experimental-mingw32>upct
png2webpll.exe olympus_xz1_01.png -o olympus_xz1_01.wbpc0 -c 0
+-----------------------------------------------------------------------
+
▌ Ultra Precision Command Timer 1.6 - Freeware (C) 1993 by Erik de
Neve ▌
+-----------------------------------------------------------------------
+
Full Command Line: png2webpll.exe olympus_xz1_01.png -o
olympus_xz1_01.wbpc0 -c 0
Starting: png2webpll.exe

png = zd, webpll = zd bytes (1.30308e-305)

Command executed: PNG2WEBPLL.EXE olympus_xz1_01.png -o
olympus_xz1_01.wbpc0 -c 0
Parameters: olympus_xz1_01.png -o olympus_xz1_01.wbpc0 -c 0
Raw total count: 1060275650 cycles on the 8253/4 programmable
interval timer.
Elapsed time: 888612086.1 microseconds = 14 minutes and 48.6121
seconds.

C:\...ibwebp-experimental-mingw32\libwebp-experimental-mingw32>upct
png2webpll.exe olympus_xz1_01.png -o olympus_xz1_01.wbpc50 -c 50
+-----------------------------------------------------------------------
+
▌ Ultra Precision Command Timer 1.6 - Freeware (C) 1993 by Erik de
Neve ▌
+-----------------------------------------------------------------------
+
Full Command Line: png2webpll.exe olympus_xz1_01.png -o
olympus_xz1_01.wbpc50 -c 50
Starting: png2webpll.exe

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc

This application has requested the Runtime to terminate it in an
unusual way.
Please contact the application's support team for more information.

Command executed: PNG2WEBPLL.EXE olympus_xz1_01.png -o
olympus_xz1_01.wbpc50 -c 50
Parameters: olympus_xz1_01.png -o olympus_xz1_01.wbpc50 -c 50
Raw total count: 1723728406 cycles on the 8253/4 programmable
interval timer.
Elapsed time: 1444648752.2 microseconds = 24 minutes and 4.6488
seconds.

---
Note: around 3 hours and 24 minutes, actually.
PNG file, just in case:
http://www.imagecompression.info/gralic/olympus_xz1_01.png (11.37 Mb)

Szabolcs Péter

unread,
Nov 20, 2011, 5:18:34 AM11/20/11
to webp-d...@webmproject.org
Thanks!

I see that the lossless encoder currently uses .webpll as extension, will it stay that way or is it just a temporary measure (so older decoders won't choke on the new files)? Also, currently the browsers able to decode .webm video are also able to decode .webp images, if the code diverges, do you plan to issue a new Content-Type-header?

SyP


"""

--
You received this message because you are subscribed to the Google Groups "WebP Discussion" group.
To post to this group, send email to webp-d...@webmproject.org.
To unsubscribe from this group, send email to webp-discuss...@webmproject.org.
For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webp-discuss/?hl=en.


Pierre Joye

unread,
Nov 20, 2011, 7:19:51 AM11/20/11
to webp-d...@webmproject.org
On Fri, Nov 18, 2011 at 5:59 AM, James Zern <jz...@google.com> wrote:

> We'll go with a mingw build for windows for the time being [7].
> You will need a bash shell to run the included script -- no batch
> file...sorry ---, but the binaries will run under the builtin command
> prompt.

Are you working on the VC makefile already or can I go ahead and fix them?

I also like to simplify it again and mimic configure options instead
of choosing configuration names, something similar to what I did for
curl:

https://github.com/bagder/curl/blob/master/winbuild/BUILD.WINDOWS.txt

Cheers,
--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

Pierre Joye

unread,
Nov 20, 2011, 8:19:49 AM11/20/11
to webp-d...@webmproject.org
hi,

Here a 1st patch (not sure anymore where is the right gerrit to send it :)

https://gist.github.com/1380260

I have an undefined symbol error about GUID_WICPixelFormat32bppRGBA,
not sure why yet, but that's only for the example, the lib builds fine
now.

James Zern

unread,
Nov 21, 2011, 5:23:31 PM11/21/11
to WebP Discussion
Hi,

On Nov 20, 5:19 am, Pierre Joye <pierre....@gmail.com> wrote:
> hi,
>
> Here a 1st patch (not sure anymore where is the right gerrit to send it :)
>
> https://gist.github.com/1380260
>

If you've already signed the contributor agreement [1] we're currently
using the gerrit server on chromium.org [2][3].

I don't think this change is necessary on the experimental branch if
you use the experimental target with the top-level Makefile.vc:
> nmake /f Makefile.vc CFG=... experimental
When run with no arguments a help will be output.

What's currently missing is anything for the lossless tree. It may be
easier to add a local Makefile.vc to lossless/, though then some work
would be needed to make portions of the top-level makefile reusable.

> I have an undefined symbol error about GUID_WICPixelFormat32bppRGBA,
> not sure why yet, but that's only for the example, the lib builds fine
> now.
>

My guess with this is that you have a dated platform sdk. A similar
issue with the mingw build was worked around by defining the GUID
using the value from 7.0a.

[1]: http://www.webmproject.org/code/contribute/
[2]: ssh://gerrit.chromium.org:29418/webm/libwebp.git
[3]: https://gerrit.chromium.org/gerrit/#q,project:webm/libwebp,n,z

James Zern

unread,
Nov 21, 2011, 5:34:01 PM11/21/11
to WebP Discussion

Thanks for the report and the file. There are quite a few unchecked
memory allocations in the experimental code, but this may be a deeper
issue.
I've opened a bug to track this:
http://code.google.com/p/webp/issues/detail?id=94

Pierre Joye

unread,
Nov 22, 2011, 3:11:59 AM11/22/11
to webp-d...@webmproject.org
hi,

On Mon, Nov 21, 2011 at 11:23 PM, James Zern <jz...@google.com> wrote:
> Hi,
>
> On Nov 20, 5:19 am, Pierre Joye <pierre....@gmail.com> wrote:
>> hi,
>>
>> Here a 1st patch (not sure anymore where is the right gerrit to send it :)
>>
>> https://gist.github.com/1380260
>>
> If you've already signed the contributor agreement [1] we're currently
> using the gerrit server on chromium.org [2][3].

Yes, did it already for the original makefile.vc contribution and some
windows fixes afair :)


> I don't think this change is necessary on the experimental branch if
> you use the experimental target with the top-level Makefile.vc:
>  > nmake /f Makefile.vc CFG=... experimental
> When run with no arguments a help will be output.

Ah right, I missed this experimental option.

> What's currently missing is anything for the lossless tree. It may be
> easier to add a local Makefile.vc to lossless/, though then some work
> would be needed to make portions of the top-level makefile reusable.

We can have all in the same Makefile.vc, as it is now. I will do it
and add the patch to gerrit.Then I will do the rewamp, it will be
easier to build/configure then as the CFG names have their limits.

>> I have an undefined symbol error about GUID_WICPixelFormat32bppRGBA,
>> not sure why yet, but that's only for the example, the lib builds fine
>> now.
>>
> My guess with this is that you have a dated platform sdk. A similar
> issue with the mingw build was worked around by defining the GUID
> using the value from 7.0a.

Cheers,--

Pascal Massimino

unread,
Nov 28, 2011, 4:44:30 PM11/28/11
to webp-d...@webmproject.org
Hi,

2011/11/20 Szabolcs Péter <syp...@gmail.com>

Thanks!

I see that the lossless encoder currently uses .webpll as extension, will it stay that way or is it just a temporary measure (so older decoders won't choke on the new files)?

that's quite a temporary measure. Eventually, the lossless bitstream will be incorporated into the RIFF structure of files, and all will be .webp.
But for now, it's easier to have a own extension that is clearly identifiable, for experimentation purpose.

Szabolcs Péter

unread,
Nov 29, 2011, 3:16:47 AM11/29/11
to webp-d...@webmproject.org
On second thought, to distinguish lossless and lossy formats by file extension would be inadequate, as - I expect - there could be both lossless and lossy images in the same stream. 

It would be nice to have an ogginfo-like tool to list the chunks in the stream, though. Or maybe any tool for RIFF streams will do, e.g. exiftool.

SyP

Pascal Massimino

unread,
Nov 29, 2011, 4:04:17 AM11/29/11
to webp-d...@webmproject.org
Hi,

2011/11/29 Szabolcs Péter <syp...@gmail.com>

On second thought, to distinguish lossless and lossy formats by file extension would be inadequate,

well, inferring information just by file extension has led to quite some traps in the past :)
Plus, because of that, most OSes are now hiding the file extensions, hehe.

 
as - I expect - there could be both lossless and lossy images in the same stream. 

It would be nice to have an ogginfo-like tool to list the chunks in the stream, though. Or maybe any tool for RIFF streams will do, e.g. exiftool.

+1
Sounds like a good starting point would be having a new flag 'dwebp -print_info' ...

Urvang Joshi

unread,
Nov 29, 2011, 4:13:05 AM11/29/11
to webp-d...@webmproject.org
Hi SyP,
Actually the "-info" option of "webpmux" binary prints something similar.
Let us know if you have any suggestions on its output.

P.S. Currently alpha & lossless chunk information is not part of that output, but we will be adding those as well sometime soon.
Reply all
Reply to author
Forward
0 new messages