QUIT makes the server crash

[Nicolas]

Hi,

I rebooted my server a second time in a row and once again it crashed after a few hours. Same reason: it crashed in ape_disconnect()

Two questions:

1) I have looked for ape_disconnect() in the source code of APE but could not find it? Where is it?

2) I ran into this description of the last modifications made to APE in Github "Fix stuck subuser when previous subuser was still writing data. Also make socket write buffer size a multiple of 2K." 

I could not find any other information here. Can someone point out why this patch was made exactly and whether it is advisable for me to download latest code and compile it (with MYSQL which may or may not work)

Thanks for your time.

Nicolas.

[Nicolas]

Noone knows where ape_disconnect() function can be found?

[Cliff]

Two questions:

1) I have looked for ape_disconnect() in the source code of APE but could not find it? Where is it?

In the source file servers.c

 

2) I ran into this description of the last modifications made to APE in Github "Fix stuck subuser when previous subuser was still writing data. Also make socket write buffer size a multiple of 2K." 

I could not find any other information here. Can someone point out why this patch was made exactly and whether it is advisable for me to download latest code and compile it (with MYSQL which may or may not work)

 

I believe this is my patch. I had several users that were halfway around the world from my servers and on very low bandwidth internet connections that were routinely causing APEServer segfault crashes. There were several different crash scenarios and some cases that only resulted in memory leakage, but I do recall that I would often experience a segfault crash soon after an intentional APEServer restart. It sounds like you might benefit from this patch. I do not use MYSQL so I cannot comment on that. I did encounter problems with the recompiled latest libape-spidermonkey that I never did resolve since I found that the 1.1.1 release version works fine for me.

 

Since these patches my APEServer has been running for 2 months now with no segfaults and no noticable memory leakage.

 

Regards,

Cliff Atwood

 

[Nicolas Guibert]

Cliff,

Many many thanks for your feedback (and your patch).

This is very very interesting.

I also get segfault a few hours after an intentional restart

I also get memory leaks (or at least memory is increasing but that may be because I have bigger and bigger arrays on my own code, don't know).

I also noticed that crashes usually happen in the night around 5-6 am (supposedly caused by remote American users, I am hosted in Europe).

Your patch really sounds interesting, but I am stuck with the Mysql problem. Probably this is not much to fix from the authors of APE, but as we all know they are now very active these days.

As I am working on a version 2 of my platform and getting closer and closer to release, I don't really see myself spending time on working around the Mysql issue as others did (calling PHP pages). This might generate unexepected surprises so I don't really want to take the risk of putting something really new like this into my version 2.

Looks like I am stuck...

Thanks again for your precious help. At least, now, I most likely know what is going on.

2012/12/31 Cliff <atw...@on-state.com>

 

--
You received this message because you are subscribed to the Google
Groups "APE Project" group.
To post to this group, send email to ape-p...@googlegroups.com
To unsubscribe from this group, send email to
ape-project...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/ape-project?hl=en
---
APE Project (Ajax Push Engine)
Official website : http://www.ape-project.org/
Git Hub : http://github.com/APE-Project/

[Peter Reijnders]

ape_disconnect is in servers.c

 

peter

[Nicolas Guibert]

Thank you Peter!

2013/1/1 Peter Reijnders <peter.ejf...@gmail.com>

ape_disconnect is in servers.c

 

peter

--

[Peter Reijnders]

Hello Nicolas

Did you find out what cause the memory leaks?
As I am reviewing the documentation, (BTW, Which is progressing rather well, I will keep you posted), I noticed some things that might be related to growing memory.

1) Possible memory leak in private properties of pipe object
    There is a comment in ape_sm_pipe_constructor
    /* TODO : This private data must be removed is the pipe is destroyed */
    It seems indeed that destroy_pipe only clears pubid
    If you have many pipes that are beeing created/delete and you have private properties that could be a problem.
    As pipes, users and channels are 'alike' I checked further
        * Ape.rmChan uses clear_properties(&chan->properties);
        * The plugin callback ape_cb_del_user uses clear_properties(&user->properties);
    It appears that a clear_properties(&pipe->properties); in pipe.c's destroy_pipe would do the trick.
    But I am not sure though. Who knows?

2) The subject of the post is "quit"; is that as in user.quit()?
    I have the feeling that the event deluser is not triggered correctly in apeuser_sm_quit (see ape_cb_del_user)
    But I am not 100 % sure. Who knows?

3) Possible memory leaks in registerCmd registerHookCmd, registerHookBadCmd.
    There are serveral comments like '/* TODO : Effacer si déjà existant (RemoveRoot & co) */'' But I think that the impact is negligable if you call them 'normally' in main.ape.js etc. I think that you have other problems than memory leaks if you keep adding these on the fly contiuously. (Would be interesting project though... :-)

Peet R.

[Nicolas]

I know this is a very old thread and not many of you are still around. But let me try my luck once more.

I can confirm that my APE server sometimes crashes a few hours after being restarted. 

Oct 13 01:21:03 server56405 kernel: [8981559.624862] traps: aped_2[19153] general protection ip:418dce sp:7ffeff989060 error:0 in aped_2[400000+2a000]

I had recompiled the code a few years ago (to increase memory) so I guess I have got the last corrections.  

The error is 0, pointing to a division by 0. Which I find a bit strange here. I don't see any division in the code below.

The ip given above leads to ape_disconnect() function from servers.c.

And more precisely to the line:

             if (co->fd == sub->client->fd) {...

I suspect the error does not come from this if statement, but rather the inside of the if.

It could be the call to http_headers_free()

Or most likely the line: deluser(sub->user, g_ape);

I don't expect a full resolution of the case and will not recompile a modified version of the code. But here are a few questions I have. If you have any clue for any of them, let me know!

1) From the general protection error line, is there any way to be more precise as to where the general protection  exactly kicks in? Can we know if it is on the if statement or below? Could it be in an inner function?

2) My best guess is that the problem is related to deleting users or sub-users. And it looks to me that users that were connected before the program was restarted find a way to interact with the new instance and make it crash (like a non-existent user asks to logout and boom!).
Do you have any clue how this can be possible?
If I knew that, I could maybe modify my client so that it does not send a request that makes a new instance crash.

Some of the C code below.

Many thanks!

static void ape_disconnect(ape_socket *co, acetables *g_ape){
subuser *sub = (subuser *)(co->attach);
if (sub != NULL) {
if (sub->wait_for_free == 1) {
free(sub);
co->attach = NULL;
return;
}
// THIS IS WHERE THE GENERAL PROTECTION KICKS IN. 

// THIS IS WHERE THE GENERAL PROTECTION KICKS IN.
// THIS IS WHERE THE GENERAL PROTECTION KICKS IN.
// THIS IS WHERE THE GENERAL PROTECTION KICKS IN.

// THIS IS WHERE THE GENERAL PROTECTION KICKS IN.  ------------>>>

if (co->fd == sub->client->fd) {                           <<<<<<<<<<-------------------------------------------------------------------------
sub->headers.sent = 0;
sub->state = ADIED;
http_headers_free(sub->headers.content);
sub->headers.content = NULL;
if (sub->user != NULL) {
if (sub->user->istmp) {
deluser(sub->user, g_ape);
co->attach = NULL;
}
}
}
}
}  

void http_headers_free(http_headers_response *headers){
struct _http_headers_fields *fields;
if (headers == NULL) {return;}
fields = headers->fields;
while(fields != NULL) {
struct _http_headers_fields *tmpfields = fields->next;
free(fields->value.val);
free(fields);
fields = tmpfields;
}
free(headers); 

}

void deluser(USERS *user, acetables *g_ape){
if (user == NULL) {
return;
}
left_all(user, g_ape);
FIRE_EVENT_NULL(deluser, user, user->istmp, g_ape);
/* kill all users connections */
clear_subusers(user, g_ape);
hashtbl_erase(g_ape->hSessid, user->sessid);
g_ape->nConnected--;
if (user->prev == NULL) {
g_ape->uHead = user->next;
} else {
user->prev->next = user->next;
}
if (user->next != NULL) {
user->next->prev = user->prev;
}
clear_sessions(user);
clear_properties(&user->properties);
destroy_pipe(user->pipe, g_ape);
/* TODO Add Event */
free(user);
}  

void left_all(USERS *user, acetables *g_ape){

CHANLIST *list, *tList;
if (user == NULL) {
return;

}
list = user->chan_foot;
while (list != NULL) {
tList = list->next;
left(user, list->chaninfo, g_ape);
list = tList;
}
} 

 

void clear_subusers(USERS *user, acetables *g_ape){
while (user->subuser != NULL) {
delsubuser(&(user->subuser), g_ape);
}
} 

 

[christian audebert]

quickly, i think you must verify if each value exist before comparing them...

i know, they must exist and be valid. BUT, it can arise that they are uninitialised, or mistaken...

verify this values exist, and contains datas and are valid.etc.. ..

just and advice. 

for myself, i stopped use APE and builded solution based on nodejs and websocket.io 

its more easy to maintain and install.. and make the job always.. 

APE was a great solution when nothing else existed, but now, its not the case.. 

--

--
You received this message because you are subscribed to the Google
Groups "APE Project" group.
To post to this group, send email to ape-p...@googlegroups.com
To unsubscribe from this group, send email to
ape-project...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/ape-project?hl=en
---
APE Project (Ajax Push Engine)
Official website : http://www.ape-project.org/
Git Hub : http://github.com/APE-Project/

---

You received this message because you are subscribed to the Google Groups "APE Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ape-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ape-project/051fe946-6512-4486-92b7-9ecdcbe1cba9n%40googlegroups.com.

[christian audebert]

sorry, i use nodejs and socket.io , breaking support to old browsers ( < ie11 for example) , but its a good thing, so many security hole in old browsers..

Le mar. 13 oct. 2020 à 12:15, Nicolas <ioa.g...@gmail.com> a écrit :

--

--
You received this message because you are subscribed to the Google
Groups "APE Project" group.
To post to this group, send email to ape-p...@googlegroups.com
To unsubscribe from this group, send email to
ape-project...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/ape-project?hl=en
---
APE Project (Ajax Push Engine)
Official website : http://www.ape-project.org/
Git Hub : http://github.com/APE-Project/

---

You received this message because you are subscribed to the Google Groups "APE Project" group.

[Nicolas]

Yes I know the best option is to switch, but I am not really sure how easy it is. Is there a way to do the transition fast?

Regarding checking if things are uninitialised, I work on a compiled program (in production) so I can't really check anything.

I can just rely on the various logs which unfortunately don't give me much clue. I activated apport to get a core dump at fail time but it did not work and I am again without a dump to look at. It seems I had the wrong configuration for this to work.

Thanks for your prompt answer my friend!

[Alok Vad]

Hi,
Thank you for your help.

--

--
You received this message because you are subscribed to the Google
Groups "APE Project" group.
To post to this group, send email to ape-p...@googlegroups.com
To unsubscribe from this group, send email to
ape-project...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/ape-project?hl=en
---
APE Project (Ajax Push Engine)
Official website : http://www.ape-project.org/
Git Hub : http://github.com/APE-Project/

---

You received this message because you are subscribed to the Google Groups "APE Project" group.

[Pablo Tejada]

Hello Nicolas,

I still can't believe you have not given up on APE yet. You are probably the only person keeping APE alive in the wild. When I first joined this project the server was very unstable so it is very hard to believe someone is still relying on this server unless they are running a really old version which is somewhat stable or are running a custom build like yourself.

I remember when I first picked up APE, at first glance and for proof of concepts the technology looked great until you tried to do something more serious. Then all of its issues and shortcomings become clearly apparent. Having to constantly restart the server to keep a somewhat stable service is far from ideal.

What site or business are you running, was it a gaming or bidding site? Anyhow, I've been looking to do some additional coding in my spare time since I don't have a good project to work of my own. Let me know if you need help coming up with a strategy to migrate your service to something like Pusher, Firebase or Socket.io

To view this discussion on the web visit https://groups.google.com/d/msgid/ape-project/CAOx4vSSg9R%2BNQa1i8uz6BOZ3mw%3DYh_kQnWMCbW5bDjagt%2B_PJA%40mail.gmail.com.

[Alok Vad]

Hello,
Thank you for helping me out with this matter.

To view this discussion on the web visit https://groups.google.com/d/msgid/ape-project/CAN64Xc4Kfn0Q8C7UTw6At21BSxRH%3DPF4T22AcyODqewDtKKGDw%40mail.gmail.com.

[Nicolas]

Hi Pablo,

Yes, we are still using it for our board gaming platform www.happymeeple.com

It is now a 10 year project so I am not sure how easy it really is to switch. I am a little bit worried. I suppose the installation of websocket/nodejs would not be the hardest part. But adapting the code on the server and client side would. We have 9000 lines of code on the server alone. Most of it is quite simple, and would not require much work but some parts would certainly be a bit problematic. I suppose the client side could be more difficult as interactions with APE are scattered everywhere and it is impossible to test every scenario. 

We rely on APE for:

- playing moves obviously

- chat (quite a few chats zones (one-on-one, by country and general chat)

- playing some AI on the server side (as well as client side), the same (game and AI) code is used on both sides.

- Mysql calls

How easy it would be to rewrite the platform for websocket/nodejs is not clear to me right now.

And APE is not that bad. Overall, it crashes maybe 5-6 times per year. It is not that unstable. It is much less since I recompiled it to increase the memory it uses (which is fixed). Before that, I reached the memory limit too quickly and boom.

I'll write an email to you with extra information.

Thanks for offering your help!

Nicolas.

[Alok Vad]

Hi,
Thank you for considering my application.
Warm Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/ape-project/36c4399b-4fca-40d3-a85c-44524f38238fn%40googlegroups.com.