Error when handling over 200 clients, probably since server sends all connected clients

[Yaron Kufert]

Hi,

I am quite new to the APE project. I had issues when over 200 clients connected to the APE server. When the client connects to the server, it issues a "CONNECT", followed by a "joinChatRoom" call. On the server, the even handler for that call looks like this:

Ape.registerCmd("joinChatRoom", true, function(params, infos) {

var cr = params.chatRoomName;

var chan = Ape.getChannelByName(cr);

    if (!$defined(chan)) {

        chan = Ape.mkChan(cr);

}

    // This data structure stores all the information about the user.

    var userState = infos.user.getProperty("userState");

    if (!userState) {

userState = {

credentials: {

watchitoo: {},

twitter: {},

facebook: {},

guest: {}

}

};

    }

    infos.user.setProperty("userState", userState);

    infos.user.join(cr);

});

the server answer looks like this:

Ape.transport.read('[{"time":"1380574741","raw":"CHANNEL","data":{"users":[{"casttype":"uni","pubid":"294fa43b43c6b152333d7d255f5496a1","properties":{"userState":{"credentials":{"watchitoo":0,"twitter":0,"facebook":0,"guest":0}}},"level":1}],"pipe":{"casttype":"multi","pubid":"e70bff78a59dcb1524750a93cf7ce943","properties":{"name":"wqd040"}}}}]')

or parsed:

[

1. {
   • "time":"1380574741",
   • "raw":"CHANNEL",
   • "data":{
     • "users":[
       1. {
          • "casttype":"uni",
          • "pubid":"294fa43b43c6b152333d7d255f5496a1",
          • "properties":{
            • "userState":{
              • "credentials":{
                • "watchitoo":0,
                • "twitter":0,
                • "facebook":0,
                • "guest":0
                }
              }
            },
          • "level":1
          }
       ],
     • "pipe":{
       • "casttype":"multi",
       • "pubid":"e70bff78a59dcb1524750a93cf7ce943",
       • "properties":{
         • "name":"wqd040"
         }
       }
     }
   }

]

As you can see, the server send the list of all connected clients, How can i stop the server from sending this list?

The problem is that once over 200 clients connected, the server sends a huge list of current users, and for some reason attach another response to it,  looking something like:

Ape.transport.read('......')Ape.transport.read('.....')Ape.transport.read('.......')

which cause the the ape client to fail on  Uncaught SyntaxError: Unexpected identifier , since it fails to execute the response. 

I would appreciate any help as I am really stuck... 

[Pablo Tejada]

Which version of the server are you using?

If your are using version v1.1.2 or above you should try using the websocket transport instead of JSONP.

Im assuming you are using JSONP by your server response, i might be wrong though.

--
--
You received this message because you are subscribed to the Google
Groups "APE Project" group.
To post to this group, send email to ape-p...@googlegroups.com
To unsubscribe from this group, send email to
ape-project...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/ape-project?hl=en
---
APE Project (Ajax Push Engine)
Official website : http://www.ape-project.org/
Git Hub : http://github.com/APE-Project/
 
---
You received this message because you are subscribed to the Google Groups "APE Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ape-project...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Yaron Kufert]

Hi, Thanks for the fast response, 

I am using version 1.01dev since i need the MySQL support, and yea, the protocol is JSONP

[Pablo Tejada]

Try to upgrade to either 1.1.2 or 1.1.3-dev they both have MySQL support, just make sure to back your current build in case something goes wrong.

[Yaron Kufert]

I've tried to build both versions over CentOS 5.8 32bit and got "undefined reference to `scramble'", so i thought that it does not support mysql . Have you manage to build it and use mysql? the gitub version? 

have you tried using it on a 64bit machine?

You received this message because you are subscribed to a topic in the Google Groups "APE Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ape-project/qV_EG6RKPys/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ape-project...@googlegroups.com.

[Pablo Tejada]

Yeah, actually my host is running on centOS 6.x 64bit and can successfully build the apr server without problems. Male you have all the required dependencies like mysql-devel

yum mysql mysql-devel

Should do, but im not if your suppose to get an specific version. On my server it just works.

[Yaron Kufert]

I am lost...

I've installed version 1.1.2 from gitub  and managed to build it successfully, but the mysql connection fails:

 ./aped: symbol lookup error: ../modules/lib/libmod_spidermonkey.so: undefined symbol: scramble

[Yaron Kufert]

from searching the web is seems that versions over 1.01dev does not support mysql..

sources:

https://groups.google.com/forum/#!topic/ape-project/a27auQs-3Yw

https://github.com/APE-Project/APE_Server/issues/25

https://github.com/APE-Project/APE_Server/issues/15

[Louis Charette]

Mysql has been fixed in version 1.1.2 and later. 

(I did, I know lol)

Envoyé de mon iPhone

[Pablo Tejada]

Can you post a list of the mysql packages installed on your machine?

[Louis Charette]

I hope this will help you. I just successfully built APE 1.1.2 on CentOS 6.4 32 bits with MySQL support. Here's the steps I used, all using root:

• Install dependencies --> yum install gcc-c++ zip mysql-devel
• Edit modules/Makefile in the APE folder you downloaded. You need to tell APE where the MySQL lib are in CentOS
• Remplace :
• MYSQL_FLAGS = -L./deps/mysac/ -I./deps/mysac/ -lmysac -lmysqlclient_r
• With:
• MYSQL_FLAGS = -L./deps/mysac/ -L/usr/lib/mysql -I./deps/mysac/ -lmysac -lmysqlclient_r
• For 64 bits, it should be:
• MYSQL_FLAGS = -L./deps/mysac/ -L/usr/lib64/mysql -I./deps/mysac/ -lmysac -lmysqlclient_r
• You can now build --> ./build.sh

Thanks to http://redrokk.com/2012/06/16/installing-ape-streaming-server-tutorial/ for the edit tip. I had this error without it (and no MySQL) : 

/usr/bin/ld: cannot find -lmysqlclient_r

Hope this will help you :-)

  - Louis

[Yaron Kufert]

Hi Louis, Thanks a lot ! this helps!

I've tried to build over 64bit centOS 6.3 but it didn't work, i got "libmod_spidermonkey.so: undefined symbol: scramble"

The MySQL packages i have are:

  [root@prov-test01 html]# rpm -qa | grep mysql

  mysql-server-5.5.28-3.el6.remi.x86_64

  mysql-libs-5.5.28-3.el6.remi.x86_64

  mysql-devel-5.5.28-3.el6.remi.x86_64

  mysql-5.5.28-3.el6.remi.x86_64

But once I've tried to install on 32bit CentOS 6.4 the installation worked like a charm :)

the bad news is that i'm back to square one - whenever a client connects he gets the list of all connected clients, which is big even if 200 users are connected... any chance to remove this?

Thanks :)

[Yaron Kufert]

I think i found out how to remove the users list sending upon user connection:

before executing build.sh, in the file `channel.c`, in the function `join`, I've marked the entire if that starts with 

    if (!(chan->flags & CHANNEL_NONINTERACTIVE))

I'm QAing it now to make sure it didn't affect anything important, but so far everything looks good. before doing so, i started getting connection errors while having 200 connections. now, after 1600 connection my load tool got stuck :)

many thanks for your help !!! 

Yaron

[Louis Charette]

I guess that would be the best solution for you if websocket doesn't help. Since it's a feature of ape I guess it's intended to send this list to everyone. It is a useful feature to built userlist when you don't want to keep track of connection/de connection   

Envoyé de mon iPhone

[Yaron Kufert]

One's feature is someone's else bug :)

I prefer using websockets (of course!!!)  but I need the system up and running as fast as i can, and i don't have the time to change the client as well at the moment... probably in the next couple of weeks. 

I was wondering, did you manage to use mysql over centOS 64bit? 

[Louis Charette]

Websocket should work out of the box if you change only the config. But it's not supported on old version of IE. 

I didn't try 64 bits yet, I need to download a new ISO. 

Envoyé de mon iPhone

[Yaron Kufert]

In that case i will try it, thanks for all of your help !!

[Pablo Tejada]

Hey,
Instead of commenting that block, can't you just use none interactive channel?

A none interactive channel name starts with a *. Ex: *testChannel. These channels wont send the user list to clients.

[Yaron Kufert]

I wasn't sure if that was the only different. I tried it and it didn't worked, but it might happened because I didn't added the '*' to all the places I needed..

[Louis Charette]

FYI, it does work with CentOS 6.4 64 bits using the steps listed in my previous message.

  - Louis

[Pablo]

Maybe i should create a new thread but i just tried to build the server on a CentOS 6.4 x64 machine.

The server builds successfully and runs ok, but MySQL doesn't work. I get the undefined symbol scramble error.

Then i modify the Makefile as suggested above and i get a different error while starting the server

[Module] Failed to load ../modules/lib/libmod_spidermonkey.so [Invalid library] (../modules/lib/libmod_spidermonkey.so: undefined symbol: SSL_connect)

[Louis Charette]

Check if there's any error during the build process. I got the scramble one at first when I built on OSX yesterday. After looking at the build error I saw there was one MySQL library missing (missing link on OSX). Maybe create a new thread with the error or detail built.

  - Louis

[Pablo Tejada]

Louis, 

Since you have the option to build the server on different systems with MySQL support you might want to update the packages site with updated binaries packages. Currently the ones on the site are builds from v1.1.1 which dont have working MySQL.

[Louis Charette]

I do have a lot of VM on my computer! I did upload a 1.1.2 64bits binaries the other day. I guess I forgot to add a link. I'll add the 32 bits later. But I can't do .debs, I don't know how that works.

Envoyé de mon iPhone

[Louis Charette]

I uploaded compiled binaries for 32 & 64 bits with MySQL support. those where built on CentOS 6.4. Can someone test those before I post them on the APE website?

http://ape-project.org/static/downloads/stable/APE_Server-1.1.2-amd64.bin.zip

http://ape-project.org/static/downloads/stable/APE_Server-1.1.2-i386.bin.zip

I don't have the knowledge for .deb package…

  - Louis

[Pablo Tejada]

using the the x64 package i get the following error:

./aped: relocation error: ../modules/lib/libmod_spidermonkey.so: symbol scramble, version libmysqlclient_16 not defined in file libmysqlclient_r.so.16 with link time reference

Also you might want to use gzip tar compression since this will be mostly use on linux machines

[Pablo Tejada]

You might also find this post on compiling generic binaries useful http://stackoverflow.com/questions/3545628/packaging-proprietary-software-for-linux/3547137#3547137 

[Louis Charette]

Could be. It was built on centOS so it's looking at the other place. I take a look at your link. 

Envoyé de mon iPhone

[Pablo Tejada]

I tried it on a CentOS 6.4 x64 machine

[Pablo]

Hello guys,

going back to the MySQL issue i was able to build and run APE with MySQL support on my CentOS x64 machine.

Now in my case is kind of a special case because i dont have root access so i have to install everything on my home directory.

It all balls down to just pointing the executable to right library path. In my case i installed mysql-devel on ~/mysql

By simply setting the LD_LIBRARY_PATH variable to the right place everything seems to work

export LD_LIBRARY_PATH=$HOME/mysql/lib

An alternative would be to add the following flag to modules/Makefile MYSQL_FLAGS

-Wl,-rpath,$HOME/mysql/lib

What that line will do is assign a path which the linker will use during the binary runtime.

Now my question is, can't we bundle the mysql client library the same way mysac is bundled?

So instead of using shared system library APE would use a static version.

[Louis Charette]

I'm far from an expert not this, but I read the source code must be bundle with ape or we must make it available or something. That would be the best solution indeed.

Envoyé de mon iPad