Rendering Streaming Youtube Videos to WPF

[Jeremiah Morrill]

Sacha made a WPF youtube viewer in WPF a little bit ago.  He used the .NET 3.5 SP 1 Webbrowser control.  Since, I have been thinking of ways to directly stream youtube right to WPF as nobody likes the airspace rules.

The first issue is that Windows does not come with a Flash movie codec (FLV directshow filter).  I installed one I found on the internet and I was able to play FLV files with the MediaElement off the HD.  Next I pointed it to an HTTP Url of an FLV and it would actually download the FLV before it would even play it.  That's crap..we want to stream it, not wait forever for it to be downloaded.  I opened up graphedit and loaded the same url and it streamed almost instantly.   Great, WMP and DShow are acting different.  This is not a big surprise I guess.  Windows Media Player 6.0 was a pure DirectShow player.  Since WMP 7 and up, Microsoft handles the http, rtsp, mms protocols and .wmv, asf files differently than it does with directshow (I have no idea why...DRM?).  Anyways since MediaElement uses WMP OCX I had to move outside MediaElement and make my own Dshow graph and used my VideoRendererElement project to render.

Here's a video of Sacha's project streaming the weezer pork and beans music video:  http://www.youtube.com/watch?v=XZ9V66vehGI
At the end I show an example of the graph that is streaming the FLV..nothing too nutty.

I still have a few hurdles to overcome, but when I get some free time, I hope to release this so folks can make some neat Youtube viewers. 

-Jer

[Sacha Barber]

very cool Jer

You rock

---

Date: Fri, 27 Jun 2008 22:23:53 -0700
From: jeremiah...@gmail.com
To: wpf-di...@googlegroups.com
Subject: Rendering Streaming Youtube Videos to WPF

---

Get Started!

[Jeremiah Morrill]

Thanks dude!  The hardest part was getting the direct url to the FLV file, which I found directions here.  My example I showed before had a hard-coded http url to the FLV.  Now that it is fixed, here is a link to a better screen capture of it...of course you see it crash near the end of the clip, heh.

-Jer

[Marlon Grech]

you're the man

--
Regards
Marlon
WPF Blog - http://marlongrech.wordpress.com/
Other Blog - http://dotnetalgos.wordpress.com/

[Jeremiah Morrill]

Wow, I noticed something even more interesting while playing random wpf videos from youtube.  Some dude made bump-map effects with the shader effects in 3.5 sp1.  Rad!  http://r.tomczak.free.fr/wordpress/index.php/2008/06/15/bump-mapping-en-wpf-35-sp1-beta/

[Sacha Barber]

Tell you what man, that makes my app look pretty cool actually.

---

Date: Fri, 27 Jun 2008 23:45:08 -0700
From: jeremiah...@gmail.com
To: wpf-di...@googlegroups.com
Subject: Re: Rendering Streaming Youtube Videos to WPF

---

Miss your Messenger buddies when on-the-go? Get Messenger on your Mobile!

[Sacha Barber]

Very cool indeed

---

Date: Fri, 27 Jun 2008 23:51:11 -0700

From: jeremiah...@gmail.com
To: wpf-di...@googlegroups.com
Subject: Re: Rendering Streaming Youtube Videos to WPF

---

Get Started!

[Jeremiah Morrill]

Thanks man...you really did all the heavy lifting though...Thats why I built off your source!   I have yet to release any of my own full projects...but it sure is fun messing with all of your guys'!

Once I tighten everything up, and package up all the necessary dshow filters, I'll send you over the code for your scrutiny. 

Right now I use some FLV demux (a/v splitter) filter and ffdshow to decode the flash video and audio.  I'd like to find a small, all in one flv filter as ffdshow can be a hefty and intrusive codec (though its the best ever!).  I also need to fix how the dshow graph is built as right now it renders to wpf and the default dshow renderer (you dont see it as its minimized in the screen cap).  This would be done by setting the filter chain to look like:  http_source->flv splitter->flv decoder->sample_grabber->null_renderer.

The downside to the whole thing is it won't support seeking.  HTTP isn't a good seeking protocol, but it can be tricked into it by using similar commands that you see used in "http download resuming" applications...though the default HTTP protocol handler does not seem to support this via dshow :(

-Jer

[Sacha Barber]

Ill look forward to seeing it Jer.


Cant say I know oo much about DirectX/DirectShow or any of that. I dont do C++ either, so that will probably be way over my head.

Still like to see it though

---

Date: Sat, 28 Jun 2008 02:19:27 -0700

---

Get 5GB of online storage for free! Get it Now!

[Jeremiah Morrill]

Interesting guys, if you add a "&fmt=18" to the end of the url for the flv, you actually get an mpeg4 h.264 stream, which is much higher resolution and quality video/audio.  This has me a little excited!

[Jeremiah Morrill]

Ok, I have the project in a state to share.  The code is mostly tidy, but it really lacks some exception handling ;).  It's kinda cool cause it plays the mpeg4 off youtube...so u get better quality than the actual site itself.

I included all the filters needed (FLV, MPEG4), so the whole zip file is 5.5 megs.  Anyone who wants a copy of it, let me know.  I have no place to host a 5 meg file as my blog won't take something that big, though gmail is happy to send it =P

-Jer

[Jeremiah Morrill]

Nevermind:  Here's the download:  http://jmorrill.hjtcentral.com/Portals/21/blog/Files/YouViewer.zip Full source included.

Just extract. 

1.)Then in the "Bin and Filters" directory, run the ffdshow installation.  Make sure the "Do not limit" option is selected in the install wizard.  This is the codec for FLV and MPEG4

2.) Run the MatroskaSplitter.  This is the demux filter for mpeg4

3.) Under and command prompt with admin privs, run "regsvr32 DShowMediaBridge.ax".  This is for the VideoRendererElement

4.) Under and command prompt with admin privs, run "regsvr32 FLVSplitter".  This is for demuxing FLV files.  Probably not needed unless you set the YouViewer to view only FLV files instead of mpeg4

5.)  Run YouViewer.exe and you can now stream youtube videos into WPF. 

I understand if nobody wants to install all these crazy binaries.  I wouldn't trust me either ;).  I promise they are not malware/spywares.

-Jer

[Pavan Podila]

This is some freaking cool stuff which should definitely be packaged as a dead simple API. Jer you should spearhead such a task ! I can't wait to do something exciting with your work, putting in 3D, Shader effects, Physics, Particle effects... an my mind is blowing up with possibilities !

Great work Jer!

--
the approach, rather than the solution
...o0O0o...
http://blog.pixelingene.com

[Jeremiah Morrill]

Sounds like fun!  I don't think I'll have much free time until middle of July though :(.  I just had to make a WPF youtube streamer because it's been on my mind for so long and I couldn't sleep last night :)

-Jer

[Sacha Barber]

This is very cool Jer

Though quite how anyone knows that much PInvoke is beyond me

PreserveSig what, Flags who, ComImport hey

---

Date: Sat, 28 Jun 2008 05:29:43 -0700

---

Messenger's gone Mobile! Get it now!

[Jeremiah Morrill]

Haha.  I thought a lot of the COM interop stuff was crazy when I first dealt with it.  Once you are forced into the programming in COM, it makes much more sense.  In the end, I was very pleased how well COM and the CLR get along together...though I do secretly wish COM would just die...

-Jer

[Marlon Grech]

COM is not that bad after all.... I played a lot with COM and I must say it is quite powerful...

[Jeremiah Morrill]

It's decent for a cross-language, component framework...But it's versioning issues, heavy reliance on the Windows registry and the apartment threading model make it feel like the crusty part of a turd sandwich.  It is much more fun consuming COM servers than writing them :)

-Jer

[Sacha Barber]

Agreed

I have done enough COM to know I hate it

---

Date: Sun, 29 Jun 2008 01:42:36 -0700

[Marlon Grech]

yea agree there :)

[Corrado Cavalli]

Does anyone knows if the new SP1 effects require Full Trust as old BitmapEffects?.

Based on my tests I presume yes, but on my SP1 build it looks like that old Bitmapeffects aren’t working anymore… L

 

Thanks

Corrado

[Jeremiah Morrill]

Everywhere that I've heard (Mix presentations, blogs) says it requires full-trust.

Odd that your old effects aren't working any more though :(

-Jer

[Corrado Cavalli]

That’s sad, being not able to use a DropShadow effect on XBap really bores me (I know there are alternatives tough…)

Corrado

[Jeremiah Morrill]

Yeah that'd bore me too...also it makes for a boring UI!

So what could be the reasoning of making pixel shaders full trust only?  I've only done a handful of shaders thus far, but it doesn't seem like there's a way to do anything malicious within them...

-Jer

[Mike Brown]

Pixel shaders run directly on the GPU and are unmanaged code. Although you can use HLSL to create a shader, you can also use GPU assembly language. Theoretically, you can do some crazy stuff that video cards were never designed to do. There are a lot of articles and sites dedicated to using the GPU as a general purpose processor. Beyond using it for physics calculations, people in the financial industy are looking to use it for complex market simulations. Read more here http://www.gpgpu.org/

[Jeremiah Morrill]

Mike, I am totally fascinated by GP-GPU projects!  Ever since I came across the Microsoft Accelerator Project.  It's really insane how fast (and highly concurrent software runs on them with their many "cores") these things are.  I came across an article just the other day how this company (cyberlink?) has a video encoder that runs on the GPU and can encode 4 1080p videos at the same time.  I have also seen folks do video-analytics (ie, object-tracking, facial detection, etc) by offloading to the GPU with amazing results.  I hope the Accelerator Project eventually evolves.  Apple is doing a huge push with OpenCL in there next OSX rev, Snow Leopard...I'd love to have a GPGPU framework built into .NET...

Speaking of "many cores"...I hope WPF will eventually benifit from multi-core CPUs...I don't know much about the internal workings or D3D, but couldn't at least the render thread gain some performance increases by running on 2+ cores?  It makes me sad to see my perty WPF app run at 25% on a quad-core, knowing that it has bottomed out.

Sorry..I'm rabbling..Been another all-weekend-nighter.

-Jer

[Mike Brown]

There's one big problem with rendering on multiple threads...synchronization. Lets say you interlace your display and have one thread handle odd lines and another handle even. What happens when one thread completes before the other? To prevent tearing, they have to synchronize. Not a problem for a somewhat static display, but when you're doing an animation, you either lose frames or your animation slows down. Neither of these are desirable.

 

So shouldn't making it multithreaded save time and be an improvement over a single thread trying to do it all? It depends. There are scenarios where yes you would get a performance boost from multi-threading. And there are edge cases where it would give a negative impact. Of course detecting those cases affects the performance of all cases.

 

I think that microsoft's approach to WPF is a good compromise...They know that the hardware still has some catching up to do. But here's the thing, I get a 3.1 for 3d Graphics on my tablet with standard on board video. While I won't be playing far cry 2 on this machine, it's still respectable. Think what happens when your average on board video gets a 5? Basically, I wouldn't worry about the effects of software rendering on performance...that's something time will address. Of course at that point there will be another pixel shader language revision and people will be concerned about the fact that Pixel Shader 3.0 effects is rendered in software...it's a never ending cycle :P