THERE HAS BEEN A 500 ERROR ON THE YOUR PLATFORM NAME HERE SERVERS

[Rainie Le]

Hi everyone, I have ran into problem with edx when i tried to access edx studio with error message like "THERE HAS BEEN A 500 ERROR ON THE YOUR PLATFORM NAME HERE SERVERS"

I saw the way to fix it on here: https://github.com/edx/configuration/wiki/Single-AWS-server-installation-using-Amazon-Machine-Image

However, it is not fixing the problem. All of my workers are working fine ( it seems). Does anyone know a way to approach the problem?

Thanks

[Rizky Ariestiyansyah]

Can you post the log file?

[Rainie Le]

Hi this is the error i got when i tried to get logs from 

vi /edx/var/log/nginx/error.log

014/09/23 13:51:37 [crit] 12514#0: *1 connect() to unix:/edx/var/forum/forum.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: forum.*, request: "PUT /api/v1/users/1?request_id=ccd2d4c0-83b1-435f-9bd7-46c989301c23 HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/api/v1/users/1?request_id=ccd2d4c0-83b1-435f-9bd7-46c989301c23", host: "localhost:18080"

2014/09/23 13:51:39 [crit] 12514#0: *3 connect() to unix:/edx/var/forum/forum.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: forum.*, request: "PUT /api/v1/users/2?request_id=e04c1263-7614-46db-90a3-302f3ec38c52 HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/api/v1/users/2?request_id=e04c1263-7614-46db-90a3-302f3ec38c52", host: "localhost:18080"

2014/09/23 13:51:40 [crit] 12514#0: *5 connect() to unix:/edx/var/forum/forum.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: forum.*, request: "PUT /api/v1/users/3?request_id=1f8c6815-bc52-4464-a8cd-a14e33a51840 HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/api/v1/users/3?request_id=1f8c6815-bc52-4464-a8cd-a14e33a51840", host: "localhost:18080"

2014/09/23 13:51:42 [crit] 12514#0: *7 connect() to unix:/edx/var/forum/forum.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: forum.*, request: "PUT /api/v1/users/4?request_id=0a512f27-bcf2-4b0c-aa2e-04bc805e1ce4 HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/api/v1/users/4?request_id=0a512f27-bcf2-4b0c-aa2e-04bc805e1ce4", host: "localhost:18080"

2014/09/23 14:41:56 [error] 27042#0: *2317 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "POST /grading_controller/login/ HTTP/1.1", upstream: "http://127.0.0.1:8060/grading_controller/login/", host: "localhost:18060"

2014/09/23 14:41:56 [error] 27042#0: *2319 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "POST /grading_controller/login/ HTTP/1.1", upstream: "http://127.0.0.1:8060/grading_controller/login/", host: "localhost:18060"

2015/11/16 08:15:50 [error] 1110#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: , request: "POST /xqueue/login/ HTTP/1.1", upstream: "http://127.0.0.1:8040/xqueue/login/", host: "localhost:18040"

2015/11/16 09:22:13 [error] 1113#0: *6039 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:22:13 [error] 1113#0: *6039 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:22:14 [error] 1113#0: *6039 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:22:16 [error] 1113#0: *6061 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "ec2-54-193-41-180.us-west-1.compute.amazonaws.com"

2015/11/16 09:26:04 [error] 1113#0: *6398 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:26:07 [error] 1113#0: *6398 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:26:16 [error] 1113#0: *6398 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:26:19 [error] 1113#0: *6398 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:27:58 [error] 1110#0: *6585 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 09:28:01 [error] 1110#0: *6585 connect() failed (111: Connection refused) while connecting to upstream, client: 107.133.150.171, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

[David Baumgold]

Looks to me like MongoDB isn’t running. You can start it by running “sudo service mongod start”. If it still doesn’t work, try removing the MongoDB lock file, like this: “sudo rm /edx/var/mongo/mongodb/mongod.lock” Then start it up again with the service command.

David Baumgold

--
You received this message because you are subscribed to the Google Groups "Open edX operations" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openedx-ops...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/1f20265e-5fa7-42c1-836d-9d3221fcecfb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rainie Le]

Hi thanks for your fast reply. I tried as you suggested and removed mongod.lock. However, there problem is still the same. Here is what I did:

ubuntu@ip-172-31-19-177:~$ sudo service mongod start

start: Job is already running: mongod

ubuntu@ip-172-31-19-177:~$ sudo rm /edx/var/mongo/mongodb/mongod.lock

ubuntu@ip-172-31-19-177:~$ sudo service mongod start

start: Job is already running: mongod

ubuntu@ip-172-31-19-177:/edx/var/mongo/mongodb$ sudo service mongod stop

mongod stop/waiting

ubuntu@ip-172-31-19-177:/edx/var/mongo/mongodb$ sudo service mongod start

mongod start/running, process 10126

However, the problem is turn into 502 Bad Gateway now when I tried to access the server on aws.

Hong

[David Baumgold]

Is the error.log file reporting a different error now?

To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/e85794fd-6e70-4ac3-a006-3e1bf846f1fe%40googlegroups.com.

[Rainie Le]

Hi,

No, the log error is still the same. Here is the latest logs:

2015/11/16 20:26:45 [error] 993#0: *13 connect() failed (111: Connection refused) while connecting to upstream, client: 130.65.86.113, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

2015/11/16 20:26:56 [error] 993#0: *13 connect() failed (111: Connection refused) while connecting to upstream, client: 130.65.86.113, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "54.193.41.180"

Hong

[David Baumgold]

Have you verified that the LMS and Studio are actually running?

From: Rainie Le <hongnh...@gmail.com>
Reply: opene...@googlegroups.com <opene...@googlegroups.com>
Date: November 16, 2015 at 4:24:54 PM
To: Open edX operations <opene...@googlegroups.com>
CC: hongnh...@gmail.com <hongnh...@gmail.com>
Subject:  Re: [openedx-ops] Re: THERE HAS BEEN A 500 ERROR ON THE YOUR PLATFORM NAME HERE SERVERS

--

You received this message because you are subscribed to the Google Groups "Open edX operations" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openedx-ops...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/474e9ae7-12db-4eb7-87fa-ebda717e1b29%40googlegroups.com.

[Rainie Le]

Hi, the studio is running even though i can edit it yet but I can see the website. The LMS is actually weird. I tried to restart the lms/cms but the status is sometimes running and sometimes starting

ubuntu@ip-172-31-19-177:/edx/var/mongo/mongodb$ sudo /edx/bin/supervisorctl restart edxapp:

edxapp:lms: stopped

edxapp:cms: stopped

edxapp:lms: started

edxapp:cms: started

ubuntu@ip-172-31-19-177:/edx/var/mongo/mongodb$ sudo /edx/bin/supervisorctl status

certs                            RUNNING   pid 22445, uptime 0:00:06

edxapp:cms                       RUNNING   pid 21605, uptime 0:01:06

edxapp:lms                       RUNNING   pid 22487, uptime 0:00:01

edxapp_worker:cms_default_4      RUNNING   pid 1483, uptime 1:05:34

edxapp_worker:cms_high_1         RUNNING   pid 1481, uptime 1:05:34

edxapp_worker:cms_low_3          RUNNING   pid 1478, uptime 1:05:34

edxapp_worker:lms_default_3      RUNNING   pid 1480, uptime 1:05:34

edxapp_worker:lms_high_4         RUNNING   pid 1484, uptime 1:05:34

edxapp_worker:lms_high_mem_2     RUNNING   pid 1479, uptime 1:05:34

edxapp_worker:lms_low_1          RUNNING   pid 1482, uptime 1:05:34

forum                            RUNNING   pid 1442, uptime 1:05:34

ora                              RUNNING   pid 1500, uptime 1:05:34

ora_celery                       FATAL     Exited too quickly (process log may have details)

xqueue                           RUNNING   pid 1448, uptime 1:05:34

xqueue_consumer                  RUNNING   pid 1441, uptime 1:05:35

ubuntu@ip-172-31-19-177:/edx/var/mongo/mongodb$ sudo /edx/bin/supervisorctl status

certs                            RUNNING   pid 22445, uptime 0:00:07

edxapp:cms                       RUNNING   pid 21605, uptime 0:01:07

edxapp:lms                       STARTING  

edxapp_worker:cms_default_4      RUNNING   pid 1483, uptime 1:05:35

edxapp_worker:cms_high_1         RUNNING   pid 1481, uptime 1:05:35

edxapp_worker:cms_low_3          RUNNING   pid 1478, uptime 1:05:35

edxapp_worker:lms_default_3      RUNNING   pid 1480, uptime 1:05:35

edxapp_worker:lms_high_4         RUNNING   pid 1484, uptime 1:05:35

edxapp_worker:lms_high_mem_2     RUNNING   pid 1479, uptime 1:05:35

edxapp_worker:lms_low_1          RUNNING   pid 1482, uptime 1:05:35

forum                            RUNNING   pid 1442, uptime 1:05:35

ora                              RUNNING   pid 1500, uptime 1:05:35

ora_celery                       FATAL     Exited too quickly (process log may have details)

xqueue                           RUNNING   pid 1448, uptime 1:05:35

xqueue_consumer                  RUNNING   pid 1441, uptime 1:05:36

[Fred Smith]

Hi,

It looks like your LMS is crashing on startup.  

The relevant log files for the LMS are:

/edx/var/log/supervisor/lms-stderr.log 

/edx/var/log/supervisor/lms-stdout.log 

/edx/var/log/lms/edx.log

You'll probably find a stack trace at the end of the first file explaining the startup error.

-Fred

To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/2459eace-9194-4d8f-b441-1a3803f9a797%40googlegroups.com.

[Rainie Le]

Hi, I figured out the problem was with my celery from the log file and after restarted, it works now.

Thanks

Hong