ACTION REQUESTED: Stress test for BigBlueButton 0.71

[Jeremy Thomerson]

Good news!  We will again be hosting a public stress test for BigBlueButton.  We will be testing the recent 0.71 release on a reference-spec dedicated server with great bandwidth.  

Please invite everyone that you can to join us.  We'd love to see 150 people or more show up.  If you have a large organization using BigBlueButton, this is your opportunity to support us.  Please have multiple employees, students, friends, aunts, uncles, nephews, grandmas, or pets show up.  The more the merrier.

When: Three days from now: Monday, November 22nd - 21:00 UTC (this is 3:00pm US/Central)

  (for converting to your time zone, see: http://tinyurl.com/22s84zv)

  Plan on the test lasting about one hour, but if you can't stay on that long, just come as much as you can, whenever you can.

Where: http://stresstest.genericconf.com/

  Visit that page and then click to join the "Official BigBlueButton Stress Test" meeting.

  Note that if you visit that page before the stress test begins, you may not see that meeting appear.

What else: the developers hosting the meeting will be on #bigbluebutton on irc.freenode.net.  If something happens (catastrophic failure, etc), you will learn about it via IRC.

Best regards,

Jeremy Thomerson

Generic Conferencing 

  in conjunction with Blindside Networks

[HostBBB.com]

Jeremy, i will try to participate... Are you testing with Asterisk or
Freeswitch?

I have a high school computer lab with 18 students earlier in the
day... They dont have cameras or mics though, and the school has
45mb down 10mb up connection

If you want to try stressing. viewing, presentation, audio, deskshare
with a full class some day let me know... anytime between 11am-1:45
eastern i can get them to login.

Stephen
hostbbb.com

On Nov 18, 7:26 pm, Jeremy Thomerson <jer...@thomersonfamily.com>
wrote:

> Good news!  We will again be hosting a public stress test for BigBlueButton.
>  We will be testing the recent 0.71 release on a reference-spec dedicated
> server with great bandwidth.
>

> *Please invite everyone that you can to join us.*  We'd love to see 150

> people or more show up.  If you have a large organization using
> BigBlueButton, this is your opportunity to support us.  Please have multiple
> employees, students, friends, aunts, uncles, nephews, grandmas, or pets show
> up.  The more the merrier.
>

> **When: Three days from now: *Monday, November 22nd - 21:00 UTC* (this is

[Jeremy Thomerson]

On Thu, Nov 18, 2010 at 9:20 PM, HostBBB.com <sd...@207me.com> wrote:

Jeremy, i will try to participate...  Are you testing with Asterisk or
Freeswitch?

This test will be with Asterisk.

 

I have a high school computer  lab with 18 students earlier in the
day...   They dont have cameras or mics though, and the school has
45mb down 10mb up connection

If you want to try stressing. viewing, presentation, audio, deskshare
with a full class some day let me know...  anytime between 11am-1:45
eastern i can get them to login.

Great!  I'll let you know.

[Andrew E]

I understand Asterisk uses less resources, but I would be interested
to see Freeswitch stress tested since the Freeswitch option is much
more practical (for the reduction in audio delay). Are there any
plans to test this configuration (maybe even with speex) as well?

I will be there for the stress test at some point. I have another
conference right before it (on our own installation of BBB!) so I
might be a little late. :-)

Andrew

On Nov 18, 8:56 pm, Jeremy Thomerson <jer...@thomersonfamily.com>
wrote:

[Andres Ederra]

Just thinking aloud...

It would be interesting to take some heaps dumps of the JVM during or after the stress test.

There are some memory leaks issues opened (http://code.google.com/p/bigbluebutton/issues/detail?id=736) that could be valuable information analysing a heap dump

But activating heap duming will affect performance? I'm not sure...

Greets

Andrés

2010/11/20 Andrew E <awen...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To post to this group, send email to bigblueb...@googlegroups.com.
To unsubscribe from this group, send email to bigbluebutton-...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/bigbluebutton-dev?hl=en.

[Denis Zgonjanin]

The stress test is going on right now. If you have a few minutes, please click on the link in the original email to join the stress test.

Hope to see you there!

[Karim Morsli - Gmail]

Denis,

 

Nice test !

Can you let me know when/ where to see some of the stats re:test results?

Thanks,

Karim

[Pablo Sotomayor]

I would apppreciate too.

Bandwidht usage, cpu usage, memory usage, disk i/o stats, etc.

[Denis Zgonjanin]

The Generic Conferencing team has some data. They will present the data when they have it ready.

Thanks to everyone who participated, and a big thanks to Generic Conf. for hosting the test, and to Jeremy for building the neat webapp to shuffle people in.

Cheers,

- Denis

[Bruce B]

I was going to ask about the webapp and the front portal. Is that to be part of the future opensource code? was neat.

[Jeremy Thomerson]

It's an open source webapp built on Wicket (the best UI framework for any complex Java application!  Okay, I'm a committer on that project, too, so I may be biased :)

https://github.com/jthomerson/bigbluebutton-gateway

It doesn't have any complex features, but it may provide you with a good base to start from.  If you clone it on GitHub and add cool features, you can send me pull requests to integrate them.  Right now the webapp keeps track of running meetings and has a background task that updates meeting stats with getMeetingInfo calls, etc...  Other than that, it's just calling create / join to allow people in progressively.  The admin page (which you get directed to when you create a meeting) has some cool features to automatically update your lists, switch someone to a moderator, allow a single person in, or allow batches of people in.  There's no database persistence because it's just meant for short-lived live meetings, so it stores meeting state in memory.

Jeremy Thomerson

http://genericconf.com

[Cameron Banta]

Hello everyone,

Here was the setup for the stress test:

Intel Core 2 Quad Q6600 (quad-core 2.4Ghz) with 4GiB RAM with Ubuntu
10.10 32bit. (This was the only server we had available at the time.)
It was a plain Ubuntu install, and I installed asterisk,
bbb-voice-conference, and bigbluebutton.
Also installed Jeremy's patched AppKonference
(https://github.com/jthomerson/AsteriskAudioKonf).
Tweaked the asterisk extensions ( to support the alpha-numeric meeting
names from Jeremy's frontend to the test ).
Changed the salt to disable the demos.

Here's what I got for the stats (sorry there's no pretty graphs this time):

> The java app was using all 4 cores, and the cpu was used at <30% the
> entire time. Total mem useage was about 728MB, of which the java
> process was using about 525MB. At peak, we were receiving 8Mbit/sec
> and sending 112MBit/sec Basically, the server never broke a sweat.

The error in the red5 log when server died was this:
2010-11-22 14:17:50,720 [http-8088-ClientPoller-2] ERROR
o.a.tomcat.util.net.NioEndpoint -
java.lang.OutOfMemoryError: GC overhead limit exceeded
2010-11-22 14:17:51,690 [pool-6736-thread-1] ERROR
o.r.s.m.InMemoryPushPushPipe - Exception when pushing message to consumer
java.lang.OutOfMemoryError: Java heap space

Thanks,
Cameron

[Scott Morris]

Hi Cameron,

Thanks for the results of the stress test.

We have done some testing and found that the out of memory exception is from Red5 not having enough heap.  In the Red5 init.d script are the options -Xms128M -Xmx128M that tell Java how much heap to allocate.  The more users connected the more memory is used and it will run until it runs out.  We have an issue open (http://code.google.com/p/bigbluebutton/issues/detail?id=758) for this and how to fix it.

Thanks for the awesome test.

Scott

[Bruce B]

What was the maximum number of participants before the server crashed?

Regards,

[Cameron Banta]

On 11/24/2010 04:17 PM, Bruce B wrote:

What was the maximum number of participants before the server crashed?

193

[Bruce B]

So, if it wasn't for the crash (the bug in Red5) would it be safe to say twice that amount could have been accommodated using that hardware specs (since the load may have only jumped to 60-70%?

Or do you think other factors may have came in?

Thanks,

[Denis Zgonjanin]

I don't think it's safe to assume that. You don't know what's beyond that point, that's just the first show stopping problem that the server ran into. There may have been others just around the corner.

- Denis

[Bruce B]

So, do you have any more tests planned yet? or would you rather wait for the bug to be worked out by Red5 team? or maybe do you want to tackle the bug? I am just excited to see *almost all of the CPU power used up to see the maximum capacity the software can be pushed at.

Regards,

[Fred Dixon]

Hi Bruce,

We'll definitely be doing more stress tests. I'm not sure that it's a
"bug" to run out of memory. Regardless of how much memory the server
has, it's going to run out at some point, and that was partly the
point of the stress test: to find that point.

The memory in this case was the small (128M) heap restriction in the
default startup values for red5. We'll be definitely increasing those
values in the next stress test. No date planned yet, watch the forums
here for another announcement from Generic Conferencing.

Regards,... Fred

[HostBBB.com]

The Audio sounded great on stress test.... Was this konference patched
with SPEEX?

Stephen

> >   in conjunction with Blindside Networks- Hide quoted text -
>
> - Show quoted text -

[Jeremy Thomerson]

No. Ulaw on Asterisk.

Jeremy Thomerson
-- sent from my "smart" phone, so please excuse spelling, formatting, or compiler errors

On Nov 25, 2010 8:05 AM, "HostBBB.com" <sd...@207me.com> wrote:

The Audio sounded great on stress test.... Was this konference patched
with SPEEX?

Stephen

On Nov 18, 9:20 pm, "HostBBB.com" <sd...@207me.com> wrote:

> Jeremy, i will try to participate...  ...

> >   in conjunction with Blindside Networks- Hide quoted text -
>
> - Show quoted text -

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" g...

[Andrew E]

What was the video quality was set to? I was watching those 20 cams
with no problems, and they looked great (not too blocky).

Andrew

On Nov 25, 7:20 am, Jeremy Thomerson <jer...@thomersonfamily.com>
wrote:

[Denis Zgonjanin]

I don't think the guys changed it, and the default is 70

[Cameron Banta]

On 11/26/2010 11:33 AM, Denis Zgonjanin wrote:

I don't think the guys changed it, and the default is 70

Yep, we didn't change the voice quality. We left most everything at the defaults for this test.