design of a media player

[BJ]

Here are some note about the design of the media player plugin.

Objective: to play media tiddlers that are linked to remote media, that are sequenced via playlists.
Non functional considerations: controls must react in real-time, ie they must feel responsive. Play must not be interrupted.
Assumption: Tiddlywiki5 should be usable on as many computers as twc, ie not just for the latest (fastest) computers.

tw5 has three devices for widget interactions, the refresh mechanism(RM), widget-tree messages(WM) and invokeActions(IA).
WMs operate from a node in the tree towards the root - for a tree with N nodes it takes a max of O(log(N)) operations to send a message from a node to
one of its ancestors. At each widget the message is either forward or examined and then possibly forwarded.
RM does a depth first transversal of the whole tree and is O(N). In addition each widget recalculates its parameters when visited.
IA are applied to only immediate children and are O(1)
So IA gives the fastest response, then WM.

Up to now user interactions (via buttons etc) result in mainly RM as changes to the ui are mediated by modifying tiddlers, on an atom based notebook there is a perceivable delay between clicking a button and the resultant action. With a mediaplayer application, delays between user actions and results are very noticeable and give a poor user experience.

The fastest design for a mediaplayer would be to have all the logic within a single widget. Players are embedded in the dom  and respond with dom events. It would be possible to have the dom 'finished playing' events directly trigger logic that would select the next track to play. Such a system would not be very flexible/extendable. It is more flexible to split the media player into logical components and to represent these components are widgets, allow a variable number of players and allowing other elements to be associated with the players (eg text and photos with mp3 tracks), as the user desires.
 
The media play can be split into a sequencer (of a playlist) and players. Players respond to events generated by the sequencer or by user interaction (eg play pause), and generate events, some of which (eg finished) need to be sent to the sequencer.

As there is one sequencer and many players, it seems logical to have the players as children of the sequencer. The device that gives the quickest possible inter-widget communication from the sequencer to the players is the IA. (Also, when action widgets are used with the button widget they separate the source of the event from the actions - this is a similar pattern). Likewise the quickest inter-widget comms from the players to the sequencer is given by WM.

Players also respond to user input - different players may different controls - (eg fastforwards for video) - having controls as children of the players allow for a natural encapsulation.

all comments wellcome!

cheers

[Arlen Beiler]

The position of the media is important, as the refresh mechanism can rerender HTML, thus halting any media that is playing. This is on the assumption that the user wants to go on and do other stuff in the tiddlywiki while he is listening to stuff. But maybe that isn't a real big deal. It should only happen when certain tiddlers get edited. I don't know much about how libraries work, but I would think that would be something to look into. At least for audio. Video would probably be more embedded within the content, but maybe someone would want that for video too. Just a thought.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

[BJ]

demo at http://mediaplayer.tiddlyspot.com/

[BJ]

If the media player is in the story then it will be interrupted if the widget tree is refreshed.I think this will only happen if a template or macro is modified. At present this results in the track restarting at the beginning, it would be better to restart at the previous position - this would need the widget to save the state of the player, but there is no support in the core to inform a widget when it is about to be removed.
If the media is only audio ( no visual element) then it may be possible to connect the audio dom element to a part of the dom outside the widget tree, but then the default controls cannot be used.

[Jed Carty]

Would creating a daemon to track the state for the player work? I am not sure how the mechanism works internally, but they can at least have internal timers that aren't affected by widgets being refreshed, so it may not have the same problems with losing information when tiddlers are rerendered that widgets do. If that is the case than you could have a media player daemon that starts when the media player is  loaded remembers the players status so any widgets can get information from it when they are forced to refresh. 

[RichShumaker]

Awesome BJ.

What types of Media do you envision playing in the playlists?

So could it play Pictures, Sounds, Videos, and SWF's?

I guess another weird off the wall question is have you ever seen Scala MM300-500?

It is software for Digital Signage and they had a stand alone version the MM series.

Thanks for making this.

Rich Shumaker

[BJ]

Thanks Jed,

a timer could be used to create a daemon and write the state to a tiddler a few times a second,  - the next issue would be that writing this data should not cause a refresh - we could have a tiddler namespace ($:/__priv__/ maybe?) that would not be refreshed.

On 24 January 2015 at 12:26, Jed Carty <inmy...@gmail.com> wrote:

Would creating a daemon to track the state for the player work? I am not sure how the mechanism works internally, but they can at least have internal timers that aren't affected by widgets being refreshed, so it may not have the same problems with losing information when tiddlers are rerendered that widgets do. If that is the case than you could have a media player daemon that starts when the media player is  loaded remembers the players status so any widgets can get information from it when they are forced to refresh. 

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywikidev/et0wVg-J6Gw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywikide...@googlegroups.com.

[BJ]

Hi Rich,

On Saturday, January 24, 2015 at 8:32:48 PM UTC-6, RichShumaker wrote:

Awesome BJ.

What types of Media do you envision playing in the playlists?

So could it play Pictures, Sounds, Videos, and SWF's?

I plan it add support for other html5 players (eg mp4 video etc).

I think I will add a  player for static content (tiddlers) which would allow the media player to act as a automated slide show. I opened an issue for it - https://github.com/buggyj/TW5-small/issues/1.

I don't know about flash (yet).

cheers

BJ

[Jeremy Ruston]

Hi BJ

Excellent summary, thank you. I'll look forward to 

> At present this results in the track restarting at the beginning, it would be better to restart at the previous position - this would need the widget to save the state of the player, but there is no support in the core to inform a widget when it is about to be removed.

I do think it would be worth extending the core such that widgets get an explicit .destroy() method.

>  the next issue would be that writing this data should not cause a refresh - we could have a tiddler namespace ($:/__priv__/ maybe?) that would not be refreshed

Ouch. We really need to get the refresh cycle 10x faster....

Best wishes

Jeremy

--

You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.

To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

--

Jeremy Ruston
mailto:jeremy...@gmail.com

[Danielo Rodríguez]

we could have a tiddler namespace ($:/__priv__/ maybe?) that would not be refreshed.

Why don't just store that data at a global structure within the TW object? For example $TW.config.mediainfo

[Tobias Beer]

Why don't just store that data at a global structure within the TW object? For example $TW.config.mediainfo

For performace reason there could perhaps be cached objects like that someday, but I guess in general there's that "everything a tiddler" paradigm.

Best wishes, Tobias. 

[BJ]

On Sunday, February 8, 2015 at 10:00:38 AM UTC-6, Danielo Rodríguez wrote:

we could have a tiddler namespace ($:/__priv__/ maybe?) that would not be refreshed.

Why don't just store that data at a global structure within the TW object? For example $TW.config.mediainfo

thanks, that is a possiblity, however as Tobias points out tw5 uses tiddlers for all it structures, and in particular the state of the ui is stored in tiddlers, and thus, for symmetry, the state of the streaming media should be in a tiddler.
Also, a tiddler prefixed $:/__priv__/ can be saved and synced thus the state is saved and it you need to stop watching a video at some point and close your tw, then when you re-open it, it would be possible to resume from where you stopped.

 

[BJ]

Added a 'staticplayer' for slideshow:

http://mediaplayer.tiddlyspot.com/#testpics

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywikidev/et0wVg-J6Gw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywikide...@googlegroups.com.