Flash safe for now: no standard video/audio codecs in HTML 5

[Karsten Silz]

Hi,

Some people thought that the upcoming HTLM 5 with standard audio and
video tags would spell the end of Flash (and Silverlight and JavaFX).
I never thought it would because these plug-ins offer much more than
just video and audio.

However, it seems now that there will be no standard audio and video
codecs in HTML 5, which means that unless a de-facto standard emerges
somewhere down the line, Flash with H.264 video will continue to
deliver video to the browser masses. For more details, see:
http://tech.slashdot.org/story/09/07/02/184251/Browser-Vendors-Force-W3C-To-Scrap-HTML-5-Codecs

In somewhat related news, XHTML 2 seems to have been canceled, making
HTML 5 the only new HTML version going forward:
http://tech.slashdot.org/story/09/07/03/1447237/XHTML-2-Cancelled

---
Karsten Silz

[Reinier Zwitserloot]

Not entirely.

if you offer just flash, you create some annoyances for your users:

- It won't work on the iPhone (major reason)
- On non-windows machines, it'll light up one CPU core, which means
notebook mac and linux users will burn through the battery.
- There's no useful right click context menu (e.g. no 'mute' in
there. There is <video> tags.

So, what I'm about to describe is not just 'to be more standards
compatible', which is good, because 'just being more standards
compatible' never made anybody do anything.

Here's what you do:

You encode your video BOTH to Ogg Theora AND h.264 via the MP4
container at 640x480 without streaming (so that its iPhone
compatible), and then:

follow the instructions at http://camendesign.com/code/video_for_everybody

This gets you a nice fallback, where the <video> tag is used offering
both ogg and h.264, which covers Safari, Firefox 3.5+, Opera10, and
Safari iPhone, as well as flash as a fallback, which covers older
versions and IE. It then falls back further, to a download link.


As its all nicely bundled up, the effort to do this is minimal, and
hosting your own video has always been quite an endeavour (you need to
figure out how to encode and all that - that's why so many people just
embed a youtube video!), so I doubt the technical difficulty of doing
this is going to stop people from adding video tag powered videos to
their websites.

On Jul 4, 4:20 pm, Karsten Silz <karsten.s...@gmail.com> wrote:
> Hi,
>
> Some people thought that the upcoming HTLM 5 with standard audio and
> video tags would spell the end of Flash (and Silverlight and JavaFX).
> I never thought it would because these plug-ins offer much more than
> just video and audio.
>
> However, it seems now that there will be no standard audio and video
> codecs in HTML 5, which means that unless a de-facto standard emerges
> somewhere down the line, Flash with H.264 video will continue to

> deliver video to the browser masses.  For more details, see:http://tech.slashdot.org/story/09/07/02/184251/Browser-Vendors-Force-...

[Joe Data]

Sure, you could "bundle Ogg Theora and H.264", but what's the
benefit? It seems that there are three reasons against using Ogg
Theora (see the email announcement at
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020620.html):
- no hardware video decoding support (cited by Apple)
- "Ogg Theora's quality-per-bit is not yet suitable for the volume
handled by YouTube" (cited by Google)
- uncertainty over whether "submarine patents" will threaten Ogg
(cited by Apple)

The last issue may or may not be real, since Apple has a vested
interest in pushing iTunes and Quicktime.

Anyway, if you encode your video to H.264, you can display it with
Flash on all computers and "natively" on the iPhone and don't worry
about these issues above. So again, what's the benefit to add Ogg
Theora? Sure, you can get the immaterial benefit of "we push open
source video", but I don't think the added expense will be worth it
for many businesses.

On Jul 4, 7:41 pm, Reinier Zwitserloot <reini...@gmail.com> wrote:
> Not entirely.
>
> if you offer just flash, you create some annoyances for your users:
>
>  - It won't work on the iPhone (major reason)
>  - On non-windows machines, it'll light up one CPU core, which means
> notebook mac and linux users will burn through the battery.
>  - There's no useful right click context menu (e.g. no 'mute' in
> there. There is <video> tags.
>
> So, what I'm about to describe is not just 'to be more standards
> compatible', which is good, because 'just being more standards
> compatible' never made anybody do anything.
>
> Here's what you do:
>
> You encode your video BOTH to Ogg Theora AND h.264 via the MP4
> container at 640x480 without streaming (so that its iPhone
> compatible), and then:
>

> follow the instructions athttp://camendesign.com/code/video_for_everybody

[Reinier Zwitserloot]

Even if you have some sort of personal vendetta against the xiph crew
and want to encode just once, using the <video> tag (with fallback to
using a flash player to render your h.264 file) has plenty of
benefits, so any perceived or real shortcomings of Ogg Theora aren't
too relevant to danger that the <video> tag presents to flash.


So, why would you double-encode:

1. (MAJOR): Because you're going to piss off firefox users. Firefox
has taken a stand and will not, as far as I understand, run <video>
tags unless there's an ogg theora source in there.

2. (ARGUABLE): Licence-free means its much easier to gain some
ubiquity; you're essentially future-proofing your content. Whether
this matters to you, or if its even a sound argument is a complex
issue that's being debated all over the interwebs and is probably not
too relevant for this forum.


However, just reason #1 (no firefox) is more than enough to spend the
extra harddisk space. harddisk space is at about 50 bucks a terabyte
these days. Unless you're youtube or vimeo, I'm having a hard time
seeing how 'I wanna save space' is going to fly as an argument. Maybe
the encode CPU time, where, last time I checked, Ogg Theora is quite a
bit slower. A lot of the patents Ogg Theora has to swerve around
involve efficient encoding tricks. That's not relevant for static
content, though.

Submarine patents: Yes, that's a problem, but note that Opera HAS gone
for it, and they are a company as well. If there are submarine patents
out there, they are expiring, and the longer you wait to come forward,
the less legal standing you have. It becomes kind of hard to honestly
claim in front of a judge that you had no clue all this stuff was
happening, and (IANAL!) there's some onus on the patent owner to
defend the patent when infractions are noticed. That's not too
relevant in a legal culture where tossing enough greenbacks on the
scales of Lady Justice will tip em, of course.

quality-per-bit: Ogg Theora is very close. Google doesn't close their
body and html tags on their pages because browsers can handle it and
it saves them 8 bytes per transfer. Sounds ridiculous until you
realize the crazy amount of traffic google's servers have to handle,
so every bit counts for them. That's a _very_ niche argument for
everybody else, though. Bandwidth isn't exactly going to bankrupt you,
these days. It's not Ogg's fault: They can't use certain tricks
because there are some bullshit patents on common techniques that they
nevertheless avoid for lack of legal funds and a solid dedication to
playing it as safe as they possibly can.


Long story short, though: Okay, then don't encode ogg. The <video> tag
is still going to put a serious dent in actual flash usage. Especially
on the fastest growing segments of the market, users will get annoyed
at lack of <video> tag based sources (netbooks and phones, which run
with non-windows OSes and have either no flash player or a very sucky
one, and Mac OS X, which as mentioned has a flash player that eats CPU
for no good reason).

I'm really interested in how google is going to roll with this. They
played ball with Apple and released an API for them to get at the
underlying sources (that's what powers the iPhone YouTube app -
obviously not a flash player). This means there's a 640x480 non-
streaming h.264 copy (or that's the actual source of all youtube
videos, I haven't checked, and it doesn't matter) for ALL youtube
videos. Why NOT stick that in a <video> tag that falls back to a flash
player? Google goes out of their way to support HTML5 and promote the
vanilla web as the app platform of the future. I'd be confused if they
don't add <video> tags to youtube soon.

[Michael Neale]

re "submarine patents" - I get the impression that is an excuse left
to last - its like invoking "national security" as a reason to not
disclose something (that everyone knows is unrelated) - a catch all
excuse, cop out etc... (I could be wrong, but it reaks of that - there
are always patents around, on everything, its quite shocking).

> >http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020620....

[Josh Suereth]

Actually, it doesn't surprise me at all.  I remember a study done using SONAR compression algorithms and comparing them to Ogg.  SONAR does a lot with compression as they deal with massive data on a routine basis.  Also, much of the data (and algorithms) are classified to protect our fleet.

As to the "unrelated" complaint, Sound Data compression is very related to national security (when used in SONAR).  Disclosing means/methods of SONAR is highly dangerous.  As an analogy, imagine if Microsoft disclosed all known vulnerabilities in windows before patching them....  right before the Black Hat convention...

I can't say any more.  (Actually I can, but It's more fun to pretend I can't.).

- Josh

[Michael Neale]

You mean you could tell us, but then you would have to kill us?

On Jul 6, 11:21 am, Josh Suereth <joshua.suer...@gmail.com> wrote:
> Actually, it doesn't surprise me at all.  I remember a study done using
> SONAR compression algorithms and comparing them to Ogg.  SONAR does a lot
> with compression as they deal with massive data on a routine basis.  Also,
> much of the data (and algorithms) are classified to protect our fleet.
>
> As to the "unrelated" complaint, Sound Data compression is very related to
> national security (when used in SONAR).  Disclosing means/methods of SONAR
> is highly dangerous.  As an analogy, imagine if Microsoft disclosed all
> known vulnerabilities in windows before patching them....  right before the
> Black Hat convention...
>
> I can't say any more.  (Actually I can, but It's more fun to pretend I
> can't.).
>
> - Josh
>

[Steven Herod]

Tell Michael, Tell Michael!

Seriously... what are plans by MS for support of these tags?

[mbien]

I see it as opportunity for proprietary technologies like silverlight
or javafx to serve as fallback mode for browsers not supporting the
video tag/codec.

It is very easy to implement and you don't have to transcode to a
different format:
http://michael-bien.com/mbien/entry/using_applets_as_fallback_mode
(this is only a sample, the security dialog would not appear if e.g
javafx was used... but javafx apparently does not support ogg too)

wikipedia for example already does that, it uses <video> in many
places.

regards,
michael

On Jul 4, 4:20 pm, Karsten Silz <karsten.s...@gmail.com> wrote:

> Hi,
>
> Some people thought that the upcoming HTLM 5 with standard audio and
> video tags would spell the end of Flash (and Silverlight and JavaFX).
> I never thought it would because these plug-ins offer much more than
> just video and audio.
>
> However, it seems now that there will be no standard audio and video
> codecs in HTML 5, which means that unless a de-facto standard emerges
> somewhere down the line, Flash with H.264 video will continue to

> deliver video to the browser masses.  For more details, see:http://tech.slashdot.org/story/09/07/02/184251/Browser-Vendors-Force-...

[Joe Data]

Some comments:
- I don't have a Mac, but I doubt that Flash in general sucks on the
Mac - after all, most Flash designers probably use the Mac, so Adobe
has an incentive.
- Bandwidth is not free, especially for high-def video, so saving
bandwidth means saving money. You not only pay for the bandwith, but
also your servers that need to be bigger to handle the more data.
- For mobile phones / netbooks / iPods / iPhones, hardware support for
video-decoding is key for long battery life, so that's a major reason
for going with H.264 which is just much better supported than Ogg
Theora (if it has any hardware decoding support at all. The iPhone
supposedly plays up to ten hours of video, and I believe this is
because it uses hardware for decoding the video (CPU? GPU?). As
Wikipidia notes, Ogg Theora performance on netbooks isn't good (http://
en.wikipedia.org/wiki/Ogg_theora#Playback_performance).
- Flash has been able to play H.264 video since the end of 2007, and
Silverlight can play it now, too. So with H.264 video you can feed
Flash players on all computers and the iPhone - and I'm sure more
mobile devices will support H.264 (like the upcoming Nvidia Tegra:
http://www.engadget.com/2009/06/04/video-nvidia-tegras-gpu-gets-busy-with-hd-video-and-full-scree/).
- Adobe is working to bring the Flash Player 10 (the same version as
on the desktop) to mobile devices and optimize them for the ARM
architecture which at least dominates the smartphones, if not most
mobile phones (http://arstechnica.com/software/news/2008/11/adobe-to-
close-desktop-mobile-flash-player-gap-with-arm-port.ars). This is
supposed to go into beta this fall and be in production next year.

Here are two more interesting articles about this - for instance, I
didn't know that Ogg Theora is based on an old commercial video codec:
http://arstechnica.com/open-source/news/2009/07/decoding-the-html-5-video-codec-debate.ars
http://www.appleinsider.com/articles/09/07/06/ogg_theora_h_264_and_the_html_5_browser_squabble.html

> >http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020620....

[Dominic Mitchell]

On Mon, Jul 06, 2009 at 08:58:05PM -0700, Joe Data wrote:
> - I don't have a Mac, but I doubt that Flash in general sucks on the
> Mac - after all, most Flash designers probably use the Mac, so Adobe
> has an incentive.

It doesn't suck per-se, but it does use a lot of CPU to display video.
I think that's the main complaint. Certainly my iMac G5 grinds to a
halt…

-Dom

[Joe Data]

I don't know a lot about Macs but this may not be Flash related, but
more of a problem that your Mac's CPU isn't powerful enough to decode
the video without assistence from the graphics card (it seems to be a
couple of years old, since it's not an Intel Mac). My Dell XPS M1710
recently mostly died (September 2006, Core Duo 2 2.3 GHz, powerful
Nvidia graphics card), and that could display 1080 HD Quicktime movies
under Windows XP. My replacement laptop (2005, Pentium M 2 GHz, ATI
chipset graphics) hardly can dispay Quicktime movie trailes at
"Medium" and chokes at most regular Youtube videos. I think you
either need a powerful CPU (dual or quad core) or hardware decoding
assistance by the graphics card to show higher quality Youtube videos
or HD video, which is an advantage for H.264 (supported in hardware
broadly) over Ogg Theora.

BTW: The Ars Technica article mentions that by the end of 2010, H.264
gets a new licensing, and it's unclear what that will look like.
Maybe this is the "Unisys moment for H.264" - Unisys had a patent that
affected GIF and sued the world and their dog for royalties; this lead
many to just switch to PNG or JPG. With H.264 supported in hardware,
such a switch to a different video format will be much harder, so I
think there'll be a lot of pressure to keep licensing fees reasonable,
but you can't rule out that this will be H.264's downfall.

[Reinier Zwitserloot]

I have a new Macbook Pro. dual core 2.4Ghz. It's _not_ a lack of CPU
power.

Also, not all flash players suck down an entire core, just most. Also,
due to thread nicing, it doesn't actually impinge on my notebook's
performance - only on other flash apps running. It does empty my
battery, get the fans going, and heat the casing quite effectively.

As to the topic of hardware decoding: Why _WONT_ flash use hardware
decoding facilities? I have no idea if it does or not, but if you know
for sure and it doesn't, and other apps like, say, VLC or even just
quicktime does, then flash again just sucks ass.

Does anyone know if putting an H.264 encoded video file on your own
webservers (instead of, say, pointing at a youtube video as a fallback
to a <video> tag with a locally hosted ogg source) requires you to
have a licence?

[Peter Becker]

With Firefox on Linux Flash can easily hog down the whole browser and it
can get noticeable outside the browser, too. That's were Flashblock
comes in even more handy -- I normally don't suffer from that problem
thanks to that little extension. It also tends to avoid the bug with
audio failing if multiple Flash videos are running (which I believe is
fixed in Flash 10, but you still see older Flash instances around).

Peter

[Bill Robertson]

Flash performance on Windows is poor too. e.g. go place a decent
flash game and watch your CPU usage. The virtual machine just isn't
nearly as good.