Play video with cocos2d-x

[guanghui8827]

Hi, everyone

Recently I have been working on the feature of playing video with cocos2d-x. Now I have implemented a VideoSprite which can accept a mp4 video file and play the video at realtime.

I use the method which reads every frame of the video at a frame rate of the video and then send the frame to the rendering buffer, in the end the GPU could rendering the image at realtime.

Now this method have three limits:

1. It’s hard to sync audio and video frame.

2. It’s not cross platform. 

3. If we play a lot of videos at the same time, it will cause performance problem.

Mingo has suggested me to use the system control to play video thus the audio and video would be at the same pace all the time.

But I think sometimes our developers need to integrate the video into part of our scene graph. 

I don’t know whether system control could achieve the same effect or not.

Any thoughts?

Regards,

Guanghui

[Xiaoming Zhang]

I think the biggest problem is how to sync audio and video. 

2. It’s not cross platform. 

I think there is method to resolve it.

Best Regards

Minggo

[Ricardo Quesada]

Hi,

What is the Use Case for the Video Player ?

By Use Case I mean, what real-world scenario triggered this requirement.

Depending on the Use Case, I can envision two possible implementations:

a)
100% integration with cocos2d-x Scene Graph

eg: To be able to scale, translate, rotate, change the Z order, run actions to the VideoPlayer node.

And in order to do so, probably we will need to do the decoding in software by an existing video decoder library.

Pros: Portable, super easy to integrate
Cons: Slow (no hardware decoder), complex to implement, possible sync issues, we need to implement our own controls (play, pause, fullscreen, letterbox, volume, etc.)

b) A simple API that plays video by wrapping the OS API to play video.

Pros: Quick to implement for one platforms (eg: iOS), fast (uses hardware decoder on iOS).

Cons: Not portable, each platform should implement the wrapper.

Which route to take depends on the Use Case.

Of course I would like to have something like a) implemented in cocos2d-x. SpriteKit has support for it, and it looks cool (although I don't know how many games are going to use the Video Player as a node). Please take a look at SKVideNode API as a reference ( https://developer.apple.com/library/ios/documentation/SpriteKit/Reference/SKVideoNode/Reference/Reference.html#//apple_ref/occ/cl/SKVideoNode ).

But as I mentioned, implementing something like a) could be very expensive (development time-wise).

If the Use Case is something like:

 Company XXX asked us to have a Video Player in cocos2d-x to play video on Android TVs, then I would go with b) and add that feature in "extensions".

--
You received this message because you are subscribed to the Google Groups "cocos2d JS development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cocos2d-js-dev...@googlegroups.com.
To post to this group, send email to cocos2d-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Zhe Wang]

The user case is actually:

1. Company XXX want to have a video player in cocos2d-x to play video on Android TV. Something looks like this

2. Some mobile games would like to play a trailer before the main menu scene. For example, Alspha 8, 秦时明月,  ...

So in this morning Minggo and me all suggest the approach (B), which is easier to develop and maintain.

[Ricardo Quesada]

Yes, for that Use Case, the b) sounds a better approach.

Since it won't support all the supported platforms, it shouldn't be part of the "core" of cocos2d-x.

Let's use this policy:

 - In order to add a feature into cocos2d-x core, it should work in all supported platforms.

If it doesn't, then we put it in the extensions folder / repository... and perhaps under the "cocos2d::experimental" namespace (or something like that).

It is a way to maintain quality: If it is part of the core then it is portable (and fast, and easy to use).
If it is not part of the "core", then we cannot guaranty portability (or speed, or ease of use).

thoughts?

[刘冠群]

Ricardo,

I've just wrapped up a Chinese email about some of my thoughts about cocos-js. I think Kai can help to translate a bit for you. 

Actually it is not related to this topic but I think maybe you could take a look.

—

Gary Liu 刘冠群

Chukong Technologies, Inc.

Coco | Cocos 2D-x | CocoaChina

m: +86-139-1000-2396

http://www.chukong-inc.com

Disclaimer: 

The information contained in this e-mail message (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended addressee, any disclosure, reproduction, distribution or other dissemination or use of this communication is strictly prohibited. If you have received this communication in error, please delete this communication and contact us by replying to this e-mail or by fax or phone immediately.

本邮件包含的信息（及其传送的附件）载有保密内容，且仅为指定收件人使用。若您并非指定收件人，请切勿披露、复印、分发或以其他方式散发或使用本电子邮件。若您误收到本电子邮件，请立即删除本电子邮件，并回复本邮件或以电话或传真方式与我们联系。

（本邮件发自iPhone）

On 2014年4月17日, at 1:43, "Ricardo Quesada" <ricardo...@gmail.com> wrote:

Yes, for that Use Case, the b) sounds a better approach.

Since it won't support all the supported platforms, it shouldn't be part of the "core" of cocos2d-x.

Let's use this policy:

 - In order to add a feature into cocos2d-x core, it should work in all supported platforms.

If it doesn't, then we put it in the extensions folder / repository... and perhaps under the "cocos2d::experimental" namespace (or something like that).

It is a way to maintain quality: If it is part of the core then it is portable (and fast, and easy to use).
If it is not part of the "core", then we cannot guaranty portability (or speed, or ease of use).

thoughts?

On 16 April 2014 10:06, Zhe Wang <wan...@chukong-inc.com> wrote:

The user case is actually:

1. Company XXX want to have a video player in cocos2d-x to play video on Android TV. Something looks like this<Screen Shot 2014-04-17 at 1.01.34 AM.png>

[Ricardo Quesada]

ok. thanks.

I'll ask Kai to translate it for me. thanks.

[Hao Wu]

There are so many usecases, 

take hearthstone for example (unity), golden cards are animated instead of a static sprite, the animation inside the card is too complex to make with particle effects, but are very short videos like 2 seconds that loops forever

[Xiaoming Zhang]

The requirement is that

* it will use UI widget to create TV UI

* it should be able to navigate to a UI widget using controller

So the our UI system should support navigation. And there should be a widget that can play a video. 

Best Regards

Minggo

On Apr 17, 2014, at 1:42 AM, Ricardo Quesada <ricardo...@gmail.com> wrote:

Yes, for that Use Case, the b) sounds a better approach.

Since it won't support all the supported platforms, it shouldn't be part of the "core" of cocos2d-x.

Let's use this policy:

 - In order to add a feature into cocos2d-x core, it should work in all supported platforms.

If it doesn't, then we put it in the extensions folder / repository... and perhaps under the "cocos2d::experimental" namespace (or something like that).

It is a way to maintain quality: If it is part of the core then it is portable (and fast, and easy to use).
If it is not part of the "core", then we cannot guaranty portability (or speed, or ease of use).

thoughts?

On 16 April 2014 10:06, Zhe Wang <wan...@chukong-inc.com> wrote:

The user case is actually:

1. Company XXX want to have a video player in cocos2d-x to play video on Android TV. Something looks like this<Screen Shot 2014-04-17 at 1.01.34 AM.png>

Best Regards

Minggo