Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Message from discussion Streaming video over serial?
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Michael J. Mahon  
View profile  
 More options Mar 30 2008, 5:32 pm
Newsgroups: comp.sys.apple2.programmer
From: "Michael J. Mahon" <mjma...@aol.com>
Date: Sun, 30 Mar 2008 14:32:09 -0700
Local: Sun, Mar 30 2008 5:32 pm
Subject: Re: Streaming video over serial?

Joshua Bell wrote:
> I have an idea for an "art" project, of sorts - streaming video from a PC
> over serial to an Apple II. The idea is to present the appearance of a very
> "retro client" version of the immersive 3D client/server application my
> company creates on the Apple - just for fun.

Sounds like great fun!

> The basic idea is as follows:

> * Service on the contemporary machine that takes frame-grabs of an
> application window at some frequency (say, 5Hz)
> * The service then processes the frames to an Apple-friendly format.
> Initially, probably uncompressed lo-res (40x48) display.
> * The data is streamed out over serial to the Apple
> * The Apple is running a fairly simple loop that reads and displays frames
> on the lo-res screen
> * The Apple is non-interactive while this is running.

Since the time to transfer a frame is certainly known as soon
as it is captured (even if it is, say, run-length compressed),
the frame rate can be made adaptive to the content.  In other
words, if it can run faster, it does.

> A lo-res screen is 40x48 or 1920 pixels/frame; 4 bits/pixel = 7680
> bits/frame. At 9600 bps that's about 1Hz refresh, which is a reasonable
> lower bound.

> Questions:
> * Can the IIc (a free one landed on my lap, which inspired this; sans power
> brick tho) do faster than 9600 reliably? Googling around, 19200bps seems
> possible (2Hz refresh) - 115k is only available on the IIgs correct?
> * Has anyone already implemented this? In 2006, Frank M. was doing
> experiments with pre-converted lo-res video (e.g.
> http://groups.google.com/group/comp.sys.apple2/browse_thread/thread/4...)
> which is an inspiration, but not directly applicable.

As David pointed out, the IIc is quite capable of 115kbps with just
a single extra POKE for configuration.

And what you propose is *very* much like video streaming from a disk
file, but with a serial link instead and possibley some decompression
code.

> Notes:
> * For maximum ease of setup, it would be ideal if the Apple side of the code
> could be bootstrapped like ADT
> * Based on David Schmenk's HBCC game, full-screen updates to the lo-res
> screen while doing lots of other processing seems entirely feasible, so I'm
> assuming that there are CPU cycles available for intra-frame compression and
> even inter-frame compression.
> * As an added feature... if the scene stops changing (to within some
> threshold), the code could switch from streaming a lo-res screen to a hi-res
> screen. This would take ~8 seconds (at 9600bps), but the result would be
> that after the scene becomes static for 8 seconds the image quality would
> increase dramatically, until the next change. To maintain the low latency,
> the hi-res stream would need be interruptable (i.e. if change is detected by
> the encoder, abort the transfer of the hi-res frame and resume lo-res
> frames) requiring a modicum of protocol design. Compression here would also
> be nice, but decompression on the Apple might be a killer.

Not if it saves more data transfer time than it costs in processor
time.  That's the tradeoff to examine, and it, of course, depends
critically on the actual serial data transfer rate.  At 115kbps, it
may be necessary to forego decompression just to maintain the UART
transfer rate.  For example, interrupt processing of UART data is
not the way to go at this rate--just simple polling and data storage.

There will be time in the loop to detect a simple protocol escape...
I'd recommend using it to trigger page-flipping at end of page and
any other things you might want to add.  If you want to add keyboard
sensing, then that will cost another 6 cycles per received character
to test and a loss of some incoming characters when a keypress is
sensed and acted upon.  A back of the envelope estimate indicates
that this will fit into the receive loop at 115kbps (~78 cycles/char).

The basic loop looks like it's about 30 cycles with a simple escape
test, so it shouldn't be critical as long as one byte value can be
dedicated to the escape function.  And even in this case, a second
consecutive "escape" could result in a return to the storage loop
to store the "escape" byte:

loop   lda  UAstat,x   ; UART have char?
        and  #mask      ; Mask status
        bxx  loop       ; -No, wait.
        lda  UAdata,x   ; -Yes, get char.
        cmp  #escape    ; Cmd escape?
        beq  cmd        ; -Yes, do it.
        sta  (zp),y     ; -No, store it
        iny             ;   and increment
        bne  nocar      ;    store address.
        inc  zp+1
nocar  lda  kbd        ; Key pressed?
        bpl  loop       ; -No.
        ...             ; -Yes, process key.
        jmp  loop       ; Return to loop, having missed x chars.
                          <Since the sending machine cannot anticipate
                           this, it will be necessary to either "flush"
                           the in-process page, for which sync has been
                           lost, or keep accurate track of how many
                           characters were lost and resume the loop with
                           Y and (zp) incremented accordingly (to just
                           re-use the previous valued for the skipped
                           bytes.>

cmd    <receive another character as above>
        <switch on char value to cmd routine>
        <each cmd routine misses x chars, where x could be
         zero for very simple commands, like page flips>

When you decide to switch to hi-res, you'll be filling the hi-res
buffer while continuing to display the last full lo-res buffer, I
presume.  So if you don't finish transferring the full hi-res
screen, the partial frame transfer would never be visible.

> * Obviously, there's the possibility of adding additional bells and
> whistles, such as letting the Apple send keystrokes back "upstream" to drive
> the source application, or sidechannel streams e.g. to display text status
> displays on the Apple, trigger beeps, etc.

Those can all be done, but will necessarily require either a "pause"
in data transmission, or, perhaps better, a pre-computed number of nulls
inserted in the stream after a protocol command to allow for processing
time on the Apple side.  Each "long" command could finish in a routine
to receive bytes until a "data restart" byte is received, which would
then return to the main loop.

For highest speed, the protocol will need to be pretty "fragile",
with recovery from a data transmission error essentially awaiting
the next re-syncing event, but it should be fine for a local serial
link.  Command decoding may need to include some redundancy to prevent
wild transfers of control (I would favor a direct vector through a
table with, perhaps, a requirement that the command byte be sent
twice, in both true and compliment form to add some robustness.)

> I haven't done any work on this and, truth be told, this is likely to be one
> of those projects that never goes anywhere. But I wanted to throw out the
> idea for comments. Is there any other interest in this?

I hope you give it a shot--it would be fun, and might open the
door to other "streaming video" activities.

-michael

NadaPong: Network game demo for Apple II computers!
Home page:  http://members.aol.com/MJMahon/

"The wastebasket is our most important design
tool--and it's seriously underused."


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.