New Internet Webcasting Software Idea

[googl...@hotpop.com]

Listening to the Linux Link Radio Show for my first time tonight, I got
to thinking about a great idea...

I think the idea below is technologically sound. We can achieve this in
2005 or 2006 with the technology we have today.

1. You want to host your own radio show.

2. You setup a website to tell when your show airs.

3. Using special software, people connect to your radio show to hear it
over compressed, telephone-quality streaming audio. The streaming audio
format can be something completely different -- far more compressed and
less technical than MP3 files, and not as good of quality.

4. Your fans, in turn, are mirrors for the show and by default their
software is set to do this if their bandwidth is detected to be high
enough. If they don't like that, they can turn this off.

5. Because your show is live, the bits are cached and then discarded.
This means that the mirrors don't have to have a lot of disk space to
carry the show. Nor do they need to push lots and lots of bits.

6. The software is set by default to be able to not only mirror the
show, but let you listen to it, as well as give you the bandwidth to do
email, gaim/irc, and some web surfing. So being a mirror doesn't
cripple you that much at all.

7. The software can also temporarily drop your mirroring ability so
that listeners can Skype into the radio show to discuss something.
Because Skype is free to Skype users, everyone pays nothing. Listeners
who were using you as a mirror hear a tiny second glitch in the sound
and their client software automatically redirects to the secondary
backup mirror that the client software had already locked in on.

So the key points of this concept are:

* As your listener base grows, so does your "bandwidth" (in a sense) --
your ability to serve the connections grows too, even exponentially. No
more hunting around for mirrors for your webcast -- your fans ARE your
mirrors.

* By starting from scratch with new software, you don't have all the
steep requirements that things like xine and mplayer have. This means
that you can have software that's easy to download and few
dependencies, making it popular.

* The advantages of Skype and other free Internet phone software
provides a free or inexpensive way for users to call in to your radio
show. You could even expand this so that if people don't have Skype,
they can use your software to call into the radio show -- and that
feature can come later.

So, I don't know how to write this kind of software. I'm just a
thinker. It's just up to the develpers out there to think about this
long enough and see if it's something they want to do.

[Julián Albo]

googl...@hotpop.com wrote:

> * By starting from scratch with new software, you don't have all the
> steep requirements that things like xine and mplayer have. This means

Yeah, you only need to write it, and make sure you write it with few
dependencies.

--
Salu2

[googl...@hotpop.com]

Oh hell yeah. Few dependencies would be a big part of this. :)

Yeah, a friend wrote me and said that this was a sensational idea and
wondered why it hadn't been implemented yet. He said it was sort of
like mixing Bit Torrent with ordinary Webcasting. Instead of paying a
lot of money to companies to be mirrors, each of your fans become
mirrors if their bandwidth is measured to support it -- unless they
uncheck that dialog box (or add a command line switch) in the software.

And this need not be considered just for web radio shows. It can be
considered for things like companies who want to do a webcast
inexpensively for a large number of users, especially startups.

Now it's only a matter of figuring out how to write the software.
Unfortunately the only code I know how to write is PHP, some limited
Python (only to meld Glade XML forms with Bash scripts), and *ack* VB6.
I've done some C++, and can read it, but I'm not trustworthy enough to
put an entire C++ project in my hands. I've heard that people can use
something like WxWindows (is that spelled right) to knock out powerful
Linux apps really fast, but I don't know if that poses dependency
issues.

Of course, one could prototype this project with Glade and use Python
to meld the PHP to Glade via background shell processes. (That's how I
made the "pgst" tool, minus using PHP.)

Specifically, the software needs to:

1. Run all functions from command line, first. Once that is achieved,
then move to putting a GUI on it. This will satisfy the GUI, non-GUI
crowds equally. This also provides the ability to get this tool out
earlier, and provides the ability for the tool to have lower
dependencies. GUIs tend to draw up dependency issues. One needs a
fallback and a command line version is ideal.

2. The software client needs to be easy to use. The end user merely
passes it a URL at a certain time of day, or schedule it with cron,
then the software does the rest.

3. About 1-2 minutes before the webcast begins, the client software
tests the bandwidth. If it's high enough and the user has not disabled
"act as mirror", then it brings up two ready-to-go streaming audio
servers and can spin out batches, two at a time, for other servers as
necessary, up to 8 servers per client, depending on bandwidth
capability. Then, the client software downloads and caches the first 10
seconds of streaming audio and makes it available on each audio server.
Then, it lets the user listen to the audio stream. While this is going
on, the audio stream is continually downloaded from a central server or
another mirror, depending on what's available out there. As it does
this, it continues to make the audio stream available on its own audio
servers that it has served up.

4. Because the clients do not all hit the central server, but may hit
the most available mirror, and because these clients in turn become
mirrors, we know that bandwidth will be utilized to the greatest
degree, spreading the load around. This lowers the bandwidth
requirement tremendously on the central server.

5. As the Internet can have traffic, the client software will
automatically recalibrate to other mirrors as necessary, providing a
continuous, non-choppy supply of sound.

And, when this is ironed out, the GUI part should be easy with Glade
and Python, as long as the dependencies are kept in check. (I recommend
building the GUI on a default copy of RH9 so that the dependencies are
low.)

Also, if the streaming audio is worked out, eventually video, even
primitive grade, could be considered the next best feature upgrade.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:

> Oh hell yeah. Few dependencies would be a big part of this. :)
>
> Yeah, a friend wrote me and said that this was a sensational idea and
> wondered why it hadn't been implemented yet. He said it was sort of
> like mixing Bit Torrent with ordinary Webcasting. Instead of paying a
> lot of money to companies to be mirrors, each of your fans become
> mirrors if their bandwidth is measured to support it -- unless they
> uncheck that dialog box (or add a command line switch) in the software.

Very interesting. I thought something like this had been
implemented already. I'm interested in the project, and I suggest
using a library for distributed computing I developed (available at
http://libglass.sf.net). This would take care of all network
dependencies, and it only uses one library (ptypes); plus it runs
in several platforms already. Something else would be required
to encode/decode the audio (and why not video too?) and output it,
but I think one or two other libraries would take care of that.
I'm in and I probably can get more help.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/
Bradley's Bromide:
If computers get too powerful, we can organize
them into a committee -- that will do them in.

[googl...@hotpop.com]

Ah, a Brazilian! I work with Brazilians in my day job for a payroll
company. You guys are big fans of open source software, especially
PostgreSQL and PHP, something I'm trying to push with my company and
we've already got a healthy foothold with that. I'm delighted to hear
your interest in my idea above.

Another thing I see with Brazilians is an interest in the KIS concept
-- Keep It Simple. That's going to be key to getting this thing off the
ground.

I took a look at Glass and it looks promising. That /DOES/ appear the
way to go.

Bruno, if your C++ skills are excellent, good, or even mediocre,
perhaps you could organize on SourceForge and take charge of the
project? I've done a tiny project at SourceForge -- pgst -- so if you
want advice on this (if you're new to that), let me know and I can
offer what I know.

Can you come up with a suggested name for the project? It's sort of
like "CastTorrent" in a sense. Perhaps a Brazil name would be suitable.
What's the Brazil word for "tornado"?

I guess the first step would be to prove that via libglass one could
get a non-streaming audio message recorded and then transfered over
TCP/IP using either APIs with low or zero dependencies, or coming up
with something new entirely. I recommend starting with Linux because it
has the largest developer base that is interested in this kind of
collaboration.

I would start with achieving 100% of the project at command line. GUIs
can always be added later. People may prefer a Mozilla, Mac, Gnome, XP,
or KDE GUI and all of these can be supported later on.

Next steps beyond that? Perhaps streaming audio, then moving on to the
P2P aspect of the project where "fans" become peer servers to the audio
stream if their bandwidth is detected as being good enough and if they
have not used a command-line switch like "--no-service".

Well, signing off for tonight. God be with you.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:

> Another thing I see with Brazilians is an interest in the KIS concept
> -- Keep It Simple. That's going to be key to getting this thing off the
> ground.

Exactly. But I prefer to add the last S to KISS :)

> I took a look at Glass and it looks promising. That /DOES/ appear the
> way to go.

Thanks...

> Bruno, if your C++ skills are excellent, good, or even mediocre,
> perhaps you could organize on SourceForge and take charge of the
> project? I've done a tiny project at SourceForge -- pgst -- so if you
> want advice on this (if you're new to that), let me know and I can
> offer what I know.

Okay, I can do that. But who's going to handle the audio
encoding/decoding? I certainly don't have time for that, do you know
anybody willing to do it??

> Can you come up with a suggested name for the project? It's sort of
> like "CastTorrent" in a sense. Perhaps a Brazil name would be suitable.
> What's the Brazil word for "tornado"?

"Tornado" :) Which is already taken in sf.net. All the names I
thought were taken, but what about bitbouncer? Portuguese words are
usually difficult to be pronounced by english speakers...

> I guess the first step would be to prove that via libglass one could
> get a non-streaming audio message recorded and then transfered over
> TCP/IP using either APIs with low or zero dependencies, or coming up
> with something new entirely. I recommend starting with Linux because it
> has the largest developer base that is interested in this kind of
> collaboration.

Well, I'm glad to say that your first step is already done! I'm
transferring images exactly like that. But I think TCP is not viable for
this project, we'll need to use UDP for sure.
I think the second step is to finished UDP support in Glass (almost
done) and design the P2P protocol. Then I can implement the protocol and
we can start testing, which is not going to be an easy thing to do...

> I would start with achieving 100% of the project at command line. GUIs
> can always be added later. People may prefer a Mozilla, Mac, Gnome, XP,
> or KDE GUI and all of these can be supported later on.

Agreed.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

You will win success in whatever calling you adopt.

[googl...@hotpop.com]

Bruno Barberi Gnecco wrote:
> > Bruno, if your C++ skills are excellent, good, or even mediocre,
> > perhaps you could organize on SourceForge and take charge of the
> > project? I've done a tiny project at SourceForge -- pgst -- so if
you
> > want advice on this (if you're new to that), let me know and I can
> > offer what I know.
>
> Okay, I can do that. But who's going to handle the audio
> encoding/decoding? I certainly don't have time for that, do you know
> anybody willing to do it??

I'll do some digging on Usenet and see if I can draw up a C++ person
willing to work on the project with the understanding that we'd like to
create a new audio encoding/decoding standard for Linux TCP and UDP
transmission with the idea of few dependencies and the largest
compatibility, using libglass if that will help.

>
> > Can you come up with a suggested name for the project? It's sort of
> > like "CastTorrent" in a sense. Perhaps a Brazil name would be
suitable.
> > What's the Brazil word for "tornado"?
>
> "Tornado" :) Which is already taken in sf.net. All the names I
> thought were taken, but what about bitbouncer? Portuguese words are
> usually difficult to be pronounced by english speakers...

What about "Spigot"? I found that Google translates this the same in
English and Portuguese. Nah. I'm sure we can come up with something
cooler. And if we tie it into "Torrent" some how, people will kind of
get the connection on how this thing sort of works.

>
> > I guess the first step would be to prove that via libglass one
could
> > get a non-streaming audio message recorded and then transfered over
> > TCP/IP using either APIs with low or zero dependencies, or coming
up
> > with something new entirely. I recommend starting with Linux
because it
> > has the largest developer base that is interested in this kind of
> > collaboration.
>
> Well, I'm glad to say that your first step is already done! I'm
> transferring images exactly like that. But I think TCP is not viable
for
> this project, we'll need to use UDP for sure.

Wow. That's a boost. I've done some socket programming in my time with
Linux PHP and also VB6. It's not the fastest thing in the world, but
hey, it runs. I've implemented a spoof Linux SMTP server that then sent
data to MS OWA, a PHP class that implemented web services, and a PHP
class that allowed one to connect to a VB6 service that used ADO to
interface with MS SQL Server.

I think UDP is a good thing to use but the problem is the packet
fragmentation and reordering that occurs when the packets are too
large. You'll have to send only small bits on UDP. (But I think you
know that from the sound of it.)

I think the original developers for TCP and UDP wanted this in the days
of Arpanet and Milnet. The goal was that TCP would be the solid
connection goal, but if not possible, UDP was the failsafe. UDP would
also be used for the heartbeat to check the connection, sending a
message from client to server to say, "Am I still running healthy on
the TCP connection?" In the event of a nuclear blast or some other
router hangup, UDP would get the packets through and then a new TCP
connection could be established on the new route.

> I think the second step is to finished UDP support in Glass (almost
> done) and design the P2P protocol. Then I can implement the protocol
and
> we can start testing, which is not going to be an easy thing to do...
>

Please share with me what you think would be a good P2P strategy. I've
kind of given some clues in a previous post of what I saw, but I'm
always willing to take better ideas.

[Måns Rullgård]

googl...@hotpop.com writes:

> Bruno Barberi Gnecco wrote:
>> > Bruno, if your C++ skills are excellent, good, or even mediocre,
>> > perhaps you could organize on SourceForge and take charge of the
>> > project? I've done a tiny project at SourceForge -- pgst -- so if you
>> > want advice on this (if you're new to that), let me know and I can
>> > offer what I know.
>>
>> Okay, I can do that. But who's going to handle the audio
>> encoding/decoding? I certainly don't have time for that, do you know
>> anybody willing to do it??
>
> I'll do some digging on Usenet and see if I can draw up a C++ person
> willing to work on the project with the understanding that we'd like to
> create a new audio encoding/decoding standard for Linux TCP and UDP
> transmission with the idea of few dependencies and the largest
> compatibility, using libglass if that will help.

You're not planning to invent a new encoding standard, are you?
Surely, the existing MPEG variants will fit most needs. You might of
course want to encapsulate the encoded data in some manner suitable
for your application.

> I think UDP is a good thing to use but the problem is the packet
> fragmentation and reordering that occurs when the packets are too
> large. You'll have to send only small bits on UDP. (But I think you
> know that from the sound of it.)

Use UDP, and limit the packet size to less than the MTU. Include some
kind of sequence number in each packet, and discard anything that
arrives late. This is the only reasonable thing to do if you want low
latency.

--
Måns Rullgård
m...@inprovide.com

[googl...@hotpop.com]

A primer on Audio Programming in C++. Evidently OSS is the way to go
because it has the largest acceptance and the fewest dependencies?

http://www.4front-tech.com/pguide/index.html

[googl...@hotpop.com]

Mans,

Don't really know whether we want a new encoding standard or not. I
need good advice here. I just don't know if using an MPEG variant, then
streaming it, will get us hung up with poor compression or large
dropouts. I also dont want to require anyone to download any more
libraries than we have to -- we want to use KISS method here and with
few dependencies.

But if MPEG variants are the way to go, then so be it.

Initially the audio could be a little less than old telephone quality,
and we can improve that later. We're thinking about a 10 second cache
before playback begins. Also, one's host for the audio could switch any
second of the broadcast if that host doesn't become available. The fan
listening to the audio would not notice a dropout, hopefully.

[googl...@hotpop.com]

I liked the name of "FanCast" because it exemplifies what we are doing
here, but that name is already taken for some sports software package
according to Google search.

So I'm stuck on the name.

[Måns Rullgård]

googl...@hotpop.com writes:

> A primer on Audio Programming in C++. Evidently OSS is the way to go
> because it has the largest acceptance and the fewest dependencies?

In Linux, ALSA is the way to go. It's included in all recent
distributions, and is much easier to work with than OSS, once you get
over the first hurdle.

--
Måns Rullgård
m...@inprovide.com

[Måns Rullgård]

googl...@hotpop.com writes:

> Mans,
>
> Don't really know whether we want a new encoding standard or not. I
> need good advice here. I just don't know if using an MPEG variant, then
> streaming it, will get us hung up with poor compression or large
> dropouts. I also dont want to require anyone to download any more
> libraries than we have to -- we want to use KISS method here and with
> few dependencies.

MPEG audio is widely supported, with several different encoders and
decoders already out there.

> But if MPEG variants are the way to go, then so be it.
>
> Initially the audio could be a little less than old telephone quality,
> and we can improve that later.

MPEG 1 layer 3 (aka MP3) at 64 kbps should be enough for that. For
better quality, increase the bitrate to 128 kbps (your average
downloaded MP3), or use AAC.

> We're thinking about a 10 second cache before playback begins. Also,
> one's host for the audio could switch any second of the broadcast if
> that host doesn't become available. The fan listening to the audio
> would not notice a dropout, hopefully.

If you want truly seamless switching between sources you'll probably
have to continuously receive from at least two sources, and if one
goes away, find another to replace it. What this does to bandwidth is
left as an exercise to the reader.

A more bandwidth friendly solution would be to only receive data from
one source, but have others "on hold" to quickly replace if something
happens.

--
Måns Rullgård
m...@inprovide.com

[googl...@hotpop.com]

Bruno, would we do good to see if a university class wants to take some
of this project on as a class assignment? Are there any university
students out there who want to take on the idea of combining Internet
streaming audio and BitTorrent (sort of) so that fans become potential
streaming hosts (unless on dial-up, or opt out with command line
switch, or bandwidth test doesn't support it).

I would think that not only would it be a fantastic problem for
students to work out in C++ on Linux, but it would be a great project
to benefit universities, if not the world.

The idea is so phenomenal that it could literally transform the web
like Skype and Vonage are beginning to. With this "P2P audio torrent"
technology in place, almost anyone could implement live or pre-recorded
audio on their website without getting hammered too badly on bandwidth.

And the next stop would be video after that, of course.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:
> Bruno, would we do good to see if a university class wants to take some
> of this project on as a class assignment? Are there any university
> students out there who want to take on the idea of combining Internet
> streaming audio and BitTorrent (sort of) so that fans become potential
> streaming hosts (unless on dial-up, or opt out with command line
> switch, or bandwidth test doesn't support it).

Sure, I see no problem at all. Anybody who wants to work is
somebody we want... But the problem is a bit more complicated than
it seems at first sight, because you need an efficient use of the
bandwidth, low lag, smart caching (specially if you allow users to
start listening/viewing from the start of the broadcasting or at
least the start of the show, which I think would be a great thing
to support), good encoding (possibly supporting two or more levels
of quality), etc.

> I would think that not only would it be a fantastic problem for
> students to work out in C++ on Linux, but it would be a great project
> to benefit universities, if not the world.

From my experience with undergraduates and their class
assignments (I have been one, heheh), they are not always (read:
practically never) thriving for the best solution. I think it would
be wise to have somebody knowledgeable supervising the sound codecs.

> The idea is so phenomenal that it could literally transform the web
> like Skype and Vonage are beginning to. With this "P2P audio torrent"
> technology in place, almost anyone could implement live or pre-recorded
> audio on their website without getting hammered too badly on bandwidth.

Yes, VoIP could be implemented as well in later stages.

> And the next stop would be video after that, of course.

What about holodecks? :)

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

Whenever a system becomes completely defined,
some damn fool discovers something which either
abolishes the system or expands it beyond recognition.

[Bruno Barberi Gnecco]

Måns Rullgård wrote:

>>We're thinking about a 10 second cache before playback begins. Also,
>>one's host for the audio could switch any second of the broadcast if
>>that host doesn't become available. The fan listening to the audio
>>would not notice a dropout, hopefully.
> If you want truly seamless switching between sources you'll probably
> have to continuously receive from at least two sources, and if one
> goes away, find another to replace it. What this does to bandwidth is
> left as an exercise to the reader.
> A more bandwidth friendly solution would be to only receive data from
> one source, but have others "on hold" to quickly replace if something
> happens.

I think your second suggestion is the way to go. The problem
of having many sources is to guarantee that they will be broadcasting
in synchrony. Adding a 5 or 10s cache, added to the tree-like
network means that some people will be received data that is several
seconds older than others, and a sudden change of source may miss
some seconds of sound. You have to compensate that, someway.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

What fools these morals be!

[googl...@hotpop.com]

I agree with your conclusion. I was just thinking about how Google
started out as a class project for two PhDs. I was thinking of more in
the PhD realm.

[googl...@hotpop.com]

Ah, good thinking guys. However, if the sound also came with an exact
microsecond queue signature in the binary audio data, one could queue
to exactly the spot they want from the second source without having to
constantly download the bytes from two sources in synchronized fashion.
The problem then becomes, if the primary host connection breaks, in
getting the second host to wake up and get the queued bytes fast enough
so that the end user doesn't notice the latency.

Anyone got ideas to expand upon that?

[Bruno Barberi Gnecco]

Ah, I see you like to think big :) PhDs would be excellent.

So, any ideas for a name? Didn't you like bitbouncer? What
about StreamTorrent?

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

If our behavior is strict, we do not need fun!

[Bernhard Agthe]

> So, any ideas for a name? Didn't you like bitbouncer? What
> about StreamTorrent?

What about StreamCat?

[googl...@hotpop.com]

Lesseee... now where can one get some really brilliant PhDs dealing
with computer audio who would be eager to do this kind of revolutionary
work?? Ah, they would probably be employed by Microsoft Research Labs
(drinking Bill Gates' bathwater). Yikes! Can't have that. For
theoretical stuff dealing with sound, I think I've seen more advanced
stuff coming from England on that sort of thing. (Take a look at the
"festival" project, which was a British project, if I'm not mistaken.)

Regarding name, StreamTorrent sounds good. StreamCat is shorter, as I
see from B. Agthe's post. I consider you the project manager, Bruno,
since you were first and have already done C++ programming with TCP/IP
on Linux. Besides, you Brazillians impress the heck out of me. I work
with 5 Brazillians now for the day job I have. So I will follow behind
your decision.

As for web design, I like to keep it short and sweet with not much
noise. I'm also not a fan of Javascript, which often breaks. I use
Javascript only when I'm backed in a corner and don't have any other
choice. Two of my favorite sites for short and sweet are target.com and
postgresql.org. I'll probably combine some elements from both for
whatever we do at SourceForge.

Well, I must return to my Linux office work over VPN. Am home with my
kid who has a cold. Talk with you in a couple days.

[Måns Rullgård]

googl...@hotpop.com writes:

> Lesseee... now where can one get some really brilliant PhDs dealing
> with computer audio who would be eager to do this kind of revolutionary
> work?? Ah, they would probably be employed by Microsoft Research Labs
> (drinking Bill Gates' bathwater). Yikes! Can't have that.

Actually, I've seen some quite interesting stuff coming out of MS
Research, for instance this:
http://detache.cmcl.cs.cmu.edu/~kunwadee/research/papers/nossdav2002-final.pdf

> For theoretical stuff dealing with sound, I think I've seen more
> advanced stuff coming from England on that sort of thing.

MS Research is in the UK.

--
Måns Rullgård
m...@inprovide.com

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:

> Regarding name, StreamTorrent sounds good. StreamCat is shorter, as I
> see from B. Agthe's post. I consider you the project manager, Bruno,

Streamcat is a trademark, streamcast is also a trademark (and
taken in sf.net). So I just registered streamtorrent.sf.net. Anybody
wanting to join the project, please send me the sf.net id.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

If you can't understand it, it is intuitively obvious.

[googl...@hotpop.com]

Wow!!! I finally got a chance to see this PDF. How funny it is that we
thought we had an original thought here, and thought we needed really
intelligent people to help with great ideas, and I make a smart remark
about MS, but sure enough MS Research has already thought about the
problem in a P2P way. And, on top of that, my comment about audio
technology research being really advanced in the UK -- well, MS
Research is in the UK!

That's pretty funny. Say, has anyone looked at this paper? It might
yield some fascinating ideas and concepts that might be applicable to
this project, even thought it's paid for by Microsoft.

[googl...@hotpop.com]

Great. Now we're cooking with grease. I'm listed as ipalmetto at
SourceForge. I'm the author of "pgst" there.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:

> As for web design, I like to keep it short and sweet with not much
> noise. I'm also not a fan of Javascript, which often breaks. I use
> Javascript only when I'm backed in a corner and don't have any other
> choice. Two of my favorite sites for short and sweet are target.com and
> postgresql.org. I'll probably combine some elements from both for
> whatever we do at SourceForge.

I agree. I suggest you setup a wiki too, to sketch new ideas,
which will be really useful while we are designing the protocol.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

Postmen never die, they just lose their zip.

[dreadful]

Sorry for drudging up an older post but what would be the difference in
this and peercast ??

MM

[googl...@hotpop.com]

Most excellent! Didn't know this already existed, MM Dreadful.

http://www.peercast.org/

[Jan Panteltje]

On a sunny day (21 Mar 2005 09:54:13 -0800) it happened googl...@hotpop.com
wrote in <1111427653.9...@o13g2000cwo.googlegroups.com>:

>Most excellent! Didn't know this already existed, MM Dreadful.
>
>http://www.peercast.org/
>

panteltje:/video/compile/peercast# ./peercast
./peercast: /usr/local/lib/libstdc++.so.5: no version information available (required by ./peercast)
'Dynamically linked' and no source code, not a good idea.
Forget it.

[Bruno Barberi Gnecco]

dreadful wrote:
> Sorry for drudging up an older post but what would be the difference in
> this and peercast ??

Some people called the attention of the group to existing
similar applications. We are checking them to see if it is worth
starting a new project...

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

Future looks spotty. You will spill soup in late evening.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:
> Most excellent! Didn't know this already existed, MM Dreadful.

Hey, where have you been, Mike? We've been wondering about
you on the streamtorrent list...

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

Violence is a sword that has no handle -- you have to hold the blade.

[googl...@hotpop.com]

> Hey, where have you been, Mike? We've been wondering about
> you on the streamtorrent list...

I'm back. Boy. I hate this new Google Groups that I use. I have to cut
and paste text myself and then do > in front of it by hand before
replying. Used to not be that way.

Well, Bruno, here's what happened...

* We hit a dry period before our rainy season and so I had to get on my
tractor to relevel the land so that water runs off. I have a water
drainage problem and my mansion is sinking slightly in the back
unfortunately. We just added a huge drain line and then gutters and now
I'm doing tractor work. Last year, we paid $19,000 to RamJack for them
to raise the house up and the scammers that work there did a half-a**
job and then asked for more cash to complete it. Unfortunately, we're
probably going to fork out more cash and have them continue, sucker
that I am. (You ever raise a 4500 sq foot pier foundation house with
bottle jacks by yourself and a wife that freaks out as you crack the
walls? It sucks.)

* We got attacked by a virus at my day job on all our Windows servers
on the SQL databases. I think it was rootkit. We now have to rebuild
the dang things. The Linux/Unix boxes just sat there, humming. I'm up
to my neck in Windows issues, one after the other. With every attack,
we're hardening and hardening the Windows boxes after each freaking
rebuild, but it's still not good enough. The attacks are relentless,
like we're at war with the North Koreans or something. (I just picked a
group of people who hate Americans.) At least with every attack I learn
some new loophole in Windows that Microsoft refuses to block and that I
have to plug by myself.

* We have security audits now and a second Windows AD migration that
we're having to go through. It's pure agony at the old day job. Wish we
had less Windows and more Linux, but my boss won't go for it unless
corporate pushes that. In the corporate office, most stuff is
converting to Linux, but much is still mainframe and Lotus Notes. At
least with mainframe apps, it's easy to make workstation changes and AD
migrations, I must admit. Wish I had that kind of environment. Ours is
different. I have a bunch of users on timebombed Delphi 3 apps we
inherited with no source code and no developers. Our new stuff is
web-based, but it's slow in coming. We won't be able to phase out the
old stuff.

Meanwhile, I've had the entrpreneurship bug again. I wish sometimes I
were bold enough to quit the day job. I've been VERY interested in
projects that can turn Linux into cash, such as:

* Asterisk PBX. My office paid $85K for the current PBX we use for 400
employees. So, I know this is a good gig if one can get a stable
Asterisk install going.

* ZoneMinder Video Surveillance (http://www.zoneminder.com/ ). My
office paid $30,000 for the video surveillance for about 8 cameras on
cables. To add 3 more, we were told it would be $13,000. Again, another
money maker to use a Linux solution instead.

* Using Linux for a timepunch system. Companies pay good money for
this, especially if it's connected to hardware like card or thumb scan.

* Using Linux for magnetic door lock control with either card or thumb
scan, with a central server to manage it all. My office paid an
outrageous amount of money for this.

So now it's a matter of getting the video surveillance demo project
going and proving to my wife that I can pull it off. If my
super-critical wife believes I can make such a startup company, then
almost anyone should. She's one tough cookie.

Let me know how the streamtorrent research is going and if peercast is
a good idea or if it's worth it to go further. I'll try to help.

God, I wish someone would create a Linux project entrepreneurship
newsgroup. Man, that would be hot!

[Jan Panteltje]

On a sunny day (29 Mar 2005 20:01:52 -0800) it happened googl...@hotpop.com
wrote in <1112153696.9...@f14g2000cwb.googlegroups.com>:

>
>* ZoneMinder Video Surveillance (http://www.zoneminder.com/ ). My
>office paid $30,000 for the video surveillance for about 8 cameras on
>cables. To add 3 more, we were told it would be $13,000. Again, another
>money maker to use a Linux solution instead.

Use webcams... motion detection, write that soft..

>* Using Linux for a timepunch system. Companies pay good money for
>this, especially if it's connected to hardware like card or thumb scan.
>
>* Using Linux for magnetic door lock control with either card or thumb
>scan, with a central server to manage it all. My office paid an
>outrageous amount of money for this.

Hey, I did that in 1983 on a IBM PC with some hardware I desiged in DOS
and it used 286 ASM (wireless cards)!

>So now it's a matter of getting the video surveillance demo project
>going and proving to my wife that I can pull it off. If my
>super-critical wife believes I can make such a startup company, then
>almost anyone should. She's one tough cookie.

Should not be impossible.
Beter if you can also read license plates.... recognize faces...
add a gun to the camera, that automaticlly fires at intruders...
all copyright ideas of cause.

[googl...@hotpop.com]

Jan,

The web cam is a great idea if you can figure out (a) how to wire it
long distances in an environment where you may have 400 employees or
more; (b) how to funnel the web cam output from more than one web cam
into a single Linux PC for collection; (c) how to get Linux to read the
video data in time slices that use less disk space, rather than the raw
video stream, and to store it with some kind of lossy compression; (d)
how to split this so that not only are you recording, but you're
providing real-time screens to security personnel.

This is what Zoneminder can do to some degree.

Now you might say, okay, go wireless, but the problem with that is that
some office security officials in large corps don't like wireless data
because it can be jammed with a jammer. So that idea is still a good
one, but only with certain customers wanting to cut costs.

For wired video streams on ordinary web cams, the problem comes down to
getting USB to transfer the signal far enough. I don't think ordinary
USB can do that, so one might need to do a USB extender via CAT 5
Ethernet. This will probably require hardware at either end of the CAT
5 to translate the signal back to USB. Is this possible? I don't know.
The way my office has this wired up is that they have what looks like
old Ethernet BNC connectors in the back of the central server. I guess
that means coaxial. Then, the coaxial cable runs very long distances to
each camera. I don't know what they use for software yet, but I can
tell you that it's Windows-based, unfortunately. If anyone has done
experiments with long USB connections, please let me know.

Sounds like you did some project like this a very long time ago with
286 ASM wireless cards. I don't know what that is per se, but it would
be great if you could share how this worked.

So, if anyone has thoughts on how to get some of this going,
hardware-wise or software-wise, please let me know. Right now,
zoneminder looks like it's worth investigating. Because I also do Linux
PHP, I also wonder if there's a PHP module that one can add on so that
they can interact with video coming in on multiple USB ports.

[Jan Panteltje]

On a sunny day (31 Mar 2005 05:59:33 -0800) it happened googl...@hotpop.com
wrote in <1112277573....@l41g2000cwc.googlegroups.com>:

>The web cam is a great idea if you can figure out (a) how to wire it
>long distances in an environment where you may have 400 employees or
>more; (b) how to funnel the web cam output from more than one web cam
>into a single Linux PC for collection; (c) how to get Linux to read the
>video data in time slices that use less disk space, rather than the raw
>video stream, and to store it with some kind of lossy compression; (d)
>how to split this so that not only are you recording, but you're
>providing real-time screens to security personnel.

I think yo ushould forget about USB and wireless for reasons you mention.
USB is a 1 meter solution.
Use ethernet, use an embedded board with the webcam to send the data as for
example mpeg4 over ethernet cabel.
Use switches.
That said, somebody (maybe reading this?) had my old mcam up on a building
site (http://panteltje.com/panteltje/mcam/ ) with a PC in Mexico, now
remote controlled, but his PC was not that stable, but not exactly next door
either....
Somebody did a FPGA board for that webcam too as study project.
If you cannot do hardware design you will have to buy something.

>
>This is what Zoneminder can do to some degree.
>
>Now you might say, okay, go wireless, but the problem with that is that

No, not wireless.

>
>For wired video streams on ordinary web cams, the problem comes down to
>getting USB to transfer the signal far enough. I don't think ordinary
>USB can do that, so one might need to do a USB extender via CAT 5
>Ethernet. This will probably require hardware at either end of the CAT

exactly.

>5 to translate the signal back to USB. Is this possible? I don't know.

No, you could use a switch and give each camera has an IP number.
Your PC input is say 100 Mbps ethernet.
This will also facilitate soem guard at home to do checks.
You need passwords / access control (cameras should not be in any way public).

>The way my office has this wired up is that they have what looks like
>old Ethernet BNC connectors in the back of the central server. I guess
>that means coaxial.

Yuk, use RJ 45 connectors / cable, with that Ethernet cable you can use 2
wires to power the embedded board / camera! There are chips specially for
that!

>Sounds like you did some project like this a very long time ago with
>286 ASM wireless cards. I don't know what that is per se, but it would
>be great if you could share how this worked.

The wireless was some XX GHz systen where you held your ID card in front of
a sensor.
It was an US made unit, that interfaced to the PC via serial port.
All IO for controlling the building automation (doors + lights etc), I
designed and was in a large double height 19 inch rack, connected to PC via
par IO cards (say flat cable).
There were also hardware components as watchdog, audio system (announcements),
sensor for light and temperature (to control sunshades) and what not.
A calling system, a clock in every office etc etc.. all controlled from the
central PC, remote burglar alarm signalling....
Ahead of time then.

'Zoneminder' as a word I dunno.
Most of these building applications I know are custom made to specs...

[googl...@hotpop.com]

Jan,

I don't think the drivers for Linux for CCTV or Video over Ethernet are
quite there yet, are they? I would think the USB video drivers would be
more readily available on Linux.

Meanwhile, check this out:

http://cpc.farnell.com/jsp/endecaSearch/partDetail.jsp?SKU=CS11725&N=401

This is a way to run USB web cams over a distance larger than 100 feet
using CAT5 Ethernet.

[pant...@yahoo.com]

Yes, but it is 99.99999999% likely NOT using ethernet protocol.
I remember those modules... that I was talking about, so I typed in
google:
mpeg security ethernet camera module
First hit is a Dutch site that has exactly what I was suggesting:
http://www.webcamcenter.nl/
(I am in the Netherlands too actually).
All your PC needs is the ethernet input, for the switch, most have that
these
days.
This is comp.os.linux.development.apps so writing the client side (not
even a
driver needed) to access TCP /UDP whatever they use, is up to you.
Look under 'software' what they have available for MS windows.
Now these guys will likely not give you info on protocol....
That is why I design my own hardware, but I would do it exactly like
this.
Almost all of the first links I find with this are useful:
http://www.connectronics.com/inscape/Video_Cameras.htm
See, it speaks TCP/IP, even knows DHCP, now that would link up
automatically.
man dhcpcd
Check those guys out, if you want to write a Linux client (like a TCP
one).
But I am sure you can google yourself too.
There did exist standalone PCB boards too.

[Bruno Barberi Gnecco]

pant...@yahoo.com wrote:

> This is comp.os.linux.development.apps so writing the client side (not
> even a
> driver needed) to access TCP /UDP whatever they use, is up to you.
> Look under 'software' what they have available for MS windows.
> Now these guys will likely not give you info on protocol....

It is probably easy to reverse engineer, though.

> That is why I design my own hardware, but I would do it exactly like
> this.

I agree. Today it's quite easy and cheap to do even much more
complex things.

> Almost all of the first links I find with this are useful:
> http://www.connectronics.com/inscape/Video_Cameras.htm
> See, it speaks TCP/IP, even knows DHCP, now that would link up
> automatically.
> man dhcpcd
> Check those guys out, if you want to write a Linux client (like a TCP
> one).
> But I am sure you can google yourself too.
> There did exist standalone PCB boards too.

You might want to check firewire cameras. I don't know how
far a firewire cable can go, but I've seen people working with them
in Linux. I'm not sure of their price, but it's certainly cheaper
than the 4.3K$ that you mentioned in the original post.

--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
http://www.lsi.usp.br/~brunobg/

Giving money and power to governments is like giving whiskey and
car keys to teenage boys.
-- P.J. O'Rourke

[googl...@hotpop.com]

The best routes I've seen so far are either:

* wired USB cameras ($25 to $35) that you stick in cheap black bubbles
(to hide from the customer that they're nothing but cheap cams) and
then use CAT6 USB extenders to get the USB to go long distances, or

* Axis 205 ($188) or VCenter IP camera NC1000 ($98) are IP cameras that
have a built-in Linux web server inside. For the Axis 205, you can
either do a screen scrape call in PHP to pull the 205's web page into a
project that centralizes all the video from the various 205's, or you
can pull just the image like this:

wget -q -m -N -nd -L -l2 -Y off --http-user=root --http-passwd=****
"http://192.168.29.46/jpg/image.jpg"

...or just show it in the PHP by some other means.

These IP cameras are of much better quality (good bang for the buck)
than the USB webcams because they support things like motion detection,
come with better wall mounts, better zoom, better video filters,
infrared night vision, and also sound.

The mpeg4 camera variety (like x10) or the older CCTV cameras are good
for solutions where someone just wants to send to a quad (or larger
multiplexer) and then out to a monitor and VCR (or DVR). They are cheap
that way in purely a hardware solution. However, they are expensive to
pull their video into Linux with TV cards. There are 4 port TV cards,
but these don't have Linux drivers as far as I've seen. The only option
is to get multiple one-port TV cards. So the cost for this is higher
when run through Linux than run through nothing but the typical
security hardware.

Aside:

My wife is losing faith in my ideas now that's she's thought about
them. She asks tough questions like:

* Okay, so let's say you build the world's cheapest but reasonable
video surveillance system that provides web access, 6 month data
retention and playback, and a way for a company manager to check on the
office or store while he's at home. If you have a day job that requires
40 hours of work, how are you going to find the time to do installs?

* How are you going to convince an existing business to ditch their
existing system and replace it with yours?

* Even if it's cheaper than anyone else in town can do this, how are
you going to convince someone to pay you $1,000 to $1,500 per each
camera in the solution?

* Obviously you're going to have to get your other ideas off the ground
like the keyless door entry, time punch, and PBX stuff off the ground
too and not just rely on this suveillance solution in order keep
afloat, right?

Spouses are great for this sort of negative thinking.

[Bruno Barberi Gnecco]

googl...@hotpop.com wrote:

> * wired USB cameras ($25 to $35) that you stick in cheap black bubbles
> (to hide from the customer that they're nothing but cheap cams) and
> then use CAT6 USB extenders to get the USB to go long distances, or

I've used cheap USB cameras that are VERY good.

> My wife is losing faith in my ideas now that's she's thought about
> them. She asks tough questions like:
>
> * Okay, so let's say you build the world's cheapest but reasonable
> video surveillance system that provides web access, 6 month data
> retention and playback, and a way for a company manager to check on the
> office or store while he's at home. If you have a day job that requires
> 40 hours of work, how are you going to find the time to do installs?

Well, even if you didn't have a day job you'd need people to
install. Installing takes time and is hard work, and I think you want
to sell more systems than you can install yourself :)

> * How are you going to convince an existing business to ditch their
> existing system and replace it with yours?

You don't. You look for new businesses or those upgrading their
systems.

> * Even if it's cheaper than anyone else in town can do this, how are
> you going to convince someone to pay you $1,000 to $1,500 per each
> camera in the solution?

You don't tell them the camera's cost a 1/10 of that :)
The truth is people *do* pay that sort of money. I was amazed
to learn that, but I think it *is* because they don't know how much
the system costs.

> * Obviously you're going to have to get your other ideas off the ground
> like the keyless door entry, time punch, and PBX stuff off the ground
> too and not just rely on this suveillance solution in order keep
> afloat, right?

This time I agree with her. I don't see what is the problem,
though.

> Spouses are great for this sort of negative thinking.

Hehehe... but a devil's advocate is useful... :)

[googl...@hotpop.com]

I've got some more details on the reason why the extra security cameras
(Google the original post) cost so much. Get this. This is an
outrageous proposal. (I kind of like it, actually, he he.) This vendor
from New Jersey is making a killing on this.

Bosch LTC-1461/20 $420 a piece
Integral DVXi 320GB DVR for 16 cams $8378
Installation material, cable, ISSI labor $6502

[googl...@hotpop.com]

When you look at Axis really close, they actually kind of suck for an 8
camera or more solution. The cheaper cameras don't even come with PoE,
so an electrician would be necessary. That's hokey. The cheaper cameras
also don't have a flexible mount, so you have to purchase that
separately and offer a kind of quirky solution. (That's probably on
purpose.)

Therefore, instead, check this one out:

http://www.pelikancam.com/cgi-bin/pelikancam/systems/psd820.html

It uses:

- 8 Samsung dome CCTV/CCD/MPEG cameras for $170 a pop
- Power-Coupled flexible RCA cables that can go 150 feet (cheap)
- RCA to BNC connectors (cheap)
- Webgateinc.com Linux-Based NDVR (but doesn't host a website)
- Free firmware updates from Webgateinc.com.
- Windows OS-based management software that connects to NDVR over
TCP/IP
- No quad, multiplexer, or sequencer is necessary -- it's all in the
NDVR
- CCTV Monitor
- Good data retention, frame rate mix

$4,000

You could sell it like the way cars and jewelry are sold these days --
put on enormous markup. You can mark it up a tremendous amount and tack
on an enormous amount for cabling and installation. Another way to
improve this solution is to build a solution that connects to this
solution and serves up everything in a PHP website. You can use
lighttpd instead of Apache just to throw off the hackers a little bit.
I read that lighttpd can host PHP 4 quite well. Then, write a script
that redirects from 80 to 8091 (or something) so that the average viral
script can't work well.

I hope we all get wealthy with Linux. Viva la Linux! :)