Hardware requirements for live streaming?

[jcisol]

Hello, we want to use mistserver for live streaming of events which might have more than a couple of viewers, let us say up to 500. What do you recommend as hardware? We expect the incoming stream to have about 0.5 - 1 Mbit/s bandwidth.
As I understand it, the crucial point is the speed of the /tmp/mist repository. Would it be a good idea to put this on a fast RAID-0 system? Bandwidth should not be an issue as we are in a large hosted environment where we can count on a 100 MBit guaranteed data rate with a physical 1 GBit connection. We plan to have 16 GB RAM so that we could easily fulfil the 0.5 MB per viewer requirement.
We would be glad to hear some comments.

Jakob Curdes

[Jaron Vietor]

Hello Jakob,

Ah, you misunderstood some of the inner workings of the server.

The /tmp/mist/ directory is mostly used to hold unix sockets - so speed of the underlying filesystem doesn't matter, and there's no need for a RAID setup or anything like that.

Live streams are stored entirely in RAM - which means any wanted DVR time will need to be available as RAM, plus the 0.5 MB per viewer.

The new HLS protocol requires more memory per viewer, by the way - about 10 seconds of video worth per viewer - which for 1Mbit would mean about 1.25MB / viewer (not a huge increase, but still worth mentioning). Other protocols still have the same requirements.

As for the hardware - 500 viewers is not that much, so most any server system should be sufficient. As a general guideline, having multiple cores helps since MistServer was designed with multiprocessing and multithreading in mind and thus highly benefits from a multicore or multiprocessor environment. Faster RAM is obviously an advantage as well, but other than that the rest of the specs do not matter that much.

Sounds like you have good specs in mind and shouldn't run into any problems.

Regards,

Jaron Viëtor

--
You received this message because you are subscribed to the Google Groups "Mistserver.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mistserver+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jakob Curdes]

Am 08.03.2013 02:00, schrieb Jaron Vietor:
> Hello Jakob,
>
> Ah, you misunderstood some of the inner workings of the server.
> The /tmp/mist/ directory is mostly used to hold unix sockets - so
> speed of the underlying filesystem doesn't matter, and there's no need
> for a RAID setup or anything like that.
> Live streams are stored entirely in RAM - which means any wanted DVR
> time will need to be available as RAM, plus the 0.5 MB per viewer.
> The new HLS protocol requires more memory per viewer, by the way -
> about 10 seconds of video worth per viewer - which for 1Mbit would
> mean about 1.25MB / viewer (not a huge increase, but still worth
> mentioning). Other protocols still have the same requirements.
>

Ah ok; I judged from recommendations of other products which recommend
RAID-0 setups for production servers in general. By DVR you mean
actually recording the stream versus just passing it to the viewer? That
would not enter into the picture as the live streams are recorded at the
upstreamer's location for later inclusion into their media library
(which will be hosted on the same server).

> As for the hardware - 500 viewers is not that much, so most any server
> system should be sufficient. As a general guideline, having multiple
> cores helps since MistServer was designed with multiprocessing and
> multithreading in mind and thus highly benefits from a multicore or
> multiprocessor environment. Faster RAM is obviously an advantage as
> well, but other than that the rest of the specs do not matter that much.
> Sounds like you have good specs in mind and shouldn't run into any
> problems.
>

Yes we have a XEON 4core CPU and 16 GB fast ECC RAM, so it looks like
this should work.

Another question; we may need a second server that runs the same stream
to a different, limited audience. Is there a way to couple the
mistservers so that they share one live upstream without investing 7500$
for the mist steward? We are prepared to buy the LTS support but the
project does not have huge amounts of money.

Thank you for answering,
Jakob Curdes

[Jaron Vietor]

On Fri, Mar 8, 2013 at 7:56 AM, Jakob Curdes <j...@info-systems.de> wrote:

Am 08.03.2013 02:00, schrieb Jaron Vietor:

Hello Jakob,

Ah, you misunderstood some of the inner workings of the server.
The /tmp/mist/ directory is mostly used to hold unix sockets - so speed of the underlying filesystem doesn't matter, and there's no need for a RAID setup or anything like that.
Live streams are stored entirely in RAM - which means any wanted DVR time will need to be available as RAM, plus the 0.5 MB per viewer.
The new HLS protocol requires more memory per viewer, by the way - about 10 seconds of video worth per viewer - which for 1Mbit would mean about 1.25MB / viewer (not a huge increase, but still worth mentioning). Other protocols still have the same requirements.

Ah ok; I judged from recommendations of other products which recommend RAID-0 setups for production servers in general. By DVR you mean actually recording the stream versus just passing it to the viewer? That would not enter into the picture as the live streams are recorded at the upstreamer's location for later inclusion into their media library (which will be hosted on the same server).

Sort-of.

DVR in this context refers to the amount a viewer can "rewind" the live stream while watching. The HLS protocol requires at least 20-30 seconds of DVR to be enabled to work at all, though approximately 1 minute (or more) is recommended. It's the size of the sliding window of media data that the server keeps buffered for clients to view.

 

As for the hardware - 500 viewers is not that much, so most any server system should be sufficient. As a general guideline, having multiple cores helps since MistServer was designed with multiprocessing and multithreading in mind and thus highly benefits from a multicore or multiprocessor environment. Faster RAM is obviously an advantage as well, but other than that the rest of the specs do not matter that much.
Sounds like you have good specs in mind and shouldn't run into any problems.

Yes we have a XEON 4core CPU and 16 GB fast ECC RAM, so it looks like this should work.

Yes, that should work just fine, no problem.

 

Another question; we may need a second server that runs the same stream to a different, limited audience. Is there a way to couple the mistservers so that they share one live upstream without investing 7500$ for the mist steward? We are prepared to buy the LTS support but the project does not have huge amounts of money.

Yes. There's several ways to connect them together manually (all the steward does is automate the process). What the best way is depends on your setup. Some examples:

- Using the MistConnRAW connector to send the raw DTSC data into the standard input of a buffer process on the other end over SSH or using netcat (advantages include better synchronization and no protocols changing the stream data anywhere in between).

- Using ffmpeg on the second server to pull the progressive FLV-based output from the first server, re-streaming it to a local live buffer over RTMP (easier to set up, but more overhead / delay involved - using a long enough DVR setting (say, 40+ seconds or so) on both servers will mask the effects of this, though).

- Piping wget of the server1 FLV progressive output into MistFLV2DTSC into a buffer process (slightly less overhead than above, otherwise mostly the same).

Many other solutions will work as well - the server was made to be flexible in usage like this.

I would recommend to put either the first or last solution mentioned above inside a while(true)-loop in a shell script for best reliability. If the connection between the servers is very good, that may not even be neccesary.

 

Thank you for answering,

No problem, and good luck :-)

 

Jakob Curdes

--
You received this message because you are subscribed to the Google Groups "Mistserver.org" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mistserver+unsubscribe@googlegroups.com.

[Jakob Curdes]

Many other solutions will work as well - the server was made to be flexible in usage like this.

I would recommend to put either the first or last solution mentioned above inside a while(true)-loop in a shell script for best reliability. If the connection between the servers is very good, that may not even be neccesary.

 

Thank you for answering,

No problem, and good luck :-)

Ok, we will do some tests and once we have put everything together I'll post our experiences.
Thank you for helping,
Jakob Curdes