Apr 5 14:58:31 ip-172-16-1-10 [service_variant=lms][django.request][env:sandbox] ERROR [ip-172-16-1-10 4480] [base.py:256] - Internal Server Error: /courses/course-v1:edX+DemoX+Demo_Course/discussion/forumTraceback (most recent call last): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/utils/decorators.py", line 145, in inner return func(*args, **kwargs) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 22, in _wrapped_view return view_func(request, *args, **kwargs) File "/edx/app/edxapp/edx-platform/lms/djangoapps/django_comment_client/forum/views.py", line 183, in wrapped_view return view_func(request, course_key, *args, **kwargs) File "/edx/app/edxapp/edx-platform/lms/djangoapps/django_comment_client/forum/views.py", line 234, in forum_form_discussion user_info = user.to_dict() File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/models.py", line 60, in to_dict self.retrieve() File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/models.py", line 65, in retrieve self._retrieve(*args, **kwargs) File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/user.py", line 138, in _retrieve metric_tags=self._metric_tags, File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/utils.py", line 86, in perform_request timeout=5 File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/requests/api.py", line 50, in request response = session.request(method=method, url=url, **kwargs) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request resp = self.send(prep, **send_kwargs) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send r = adapter.send(request, **kwargs) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/requests/adapters.py", line 415, in send raise ConnectionError(err, request=request)ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))Apr 5 14:58:31 ip-172-16-1-10 [service_variant=lms][edx.footer][env:sandbox] INFO [ip-172-16-1-10 4480] [api.py:282] - Failed to find footer logo at 'themes/edx.org/images/logo.png', using 'images/logo.png' instead
==> nginx/error.log <==2016/04/05 15:00:23 [crit] 18999#0: *708 connect() to unix:/edx/var/forum/forum.sock failed (2: No such file or directory) while connecting to upstream, client: 173.184.233.21, server: forum.*, request: "GET /favicon.ico HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/favicon.ico", host: "forum. mydomain.org", referrer: "http://forum. mydomain.org/"
==> nginx/error.log <==2016/04/05 15:11:42 [error] 18999#0: *859 connect() to unix:/edx/var/forum/forum.sock failed (111: Connection refused) while connecting to upstream, client: 173.184.232.9, server: forum.*, request: "GET / HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/", host: "forum.mydomain.org"
==> nginx/error.log <==2016/04/05 15:11:42 [error] 18999#0: *859 connect() to unix:/edx/var/forum/forum.sock failed (111: Connection refused) while connecting to upstream, client: 173.184.232.9, server: forum.*, request: "GET /favicon.ico HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/favicon.ico", host: "forum.mydomain.org", referrer: "http://forum.mydomain.org/"
upstream forum_app_server { server unix:/edx/var/forum/forum.sock fail_timeout=0;}
server {
server_name forum.*; listen 80 ; client_max_body_size 1M; keepalive_timeout 5;
location / { try_files $uri @proxy_to_app; }
location @proxy_to_app { proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host $http_host;
proxy_redirect off; proxy_pass http://forum_app_server; }}
COMMENTS_SERVICE_KEY: passwordCOMMENTS_SERVICE_URL: http://forum.mydomain.org
==> nginx/error.log <==2016/04/06 01:03:33 [error] 29653#0: *135 connect() to unix:/edx/var/forum/forum.sock failed (111: Connection refused) while connecting to upstream, client: 173.184.240.9, server: forum.*, request: "GET / HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/", host: "forum.mydomain.org"==> nginx/error.log <==2016/04/06 01:03:34 [error] 29653#0: *135 connect() to unix:/edx/var/forum/forum.sock failed (111: Connection refused) while connecting to upstream, client: 173.184.240.9, server: forum.*, request: "GET /favicon.ico HTTP/1.1", upstream: "http://unix:/edx/var/forum/forum.sock:/favicon.ico", host: "forum. mydomain.org", referrer: "http://forum.mydomain.org/"
analytics_api RUNNING pid 22831, uptime 0:19:48certs RUNNING pid 29669, uptime 0:15:18ecommerce RUNNING pid 20879, uptime 0:20:08ecomworker RUNNING pid 21006, uptime 0:20:06edxapp:cms RUNNING pid 19057, uptime 0:21:44edxapp:lms RUNNING pid 19414, uptime 0:21:33edxapp_worker:cms_default_1 RUNNING pid 9764, uptime 0:34:37edxapp_worker:cms_high_1 RUNNING pid 10119, uptime 0:34:33edxapp_worker:cms_low_1 RUNNING pid 10138, uptime 0:34:30edxapp_worker:lms_default_1 RUNNING pid 10154, uptime 0:34:26edxapp_worker:lms_high_1 RUNNING pid 10177, uptime 0:34:22edxapp_worker:lms_high_mem_1 RUNNING pid 10200, uptime 0:34:18edxapp_worker:lms_low_1 RUNNING pid 10226, uptime 0:34:14forum RUNNING pid 5880, uptime 0:00:19insights RUNNING pid 25300, uptime 0:16:45notifier-celery-workers RUNNING pid 29730, uptime 0:15:06notifier-scheduler RUNNING pid 29681, uptime 0:15:17xqueue RUNNING pid 28718, uptime 0:15:35xqueue_consumer RUNNING pid 28978, uptime 0:15:33
forum-sup 9303 www-data cwd DIR 202,1 4096 5120521 /edx/app/forum/cs_comments_serviceruby 9335 www-data cwd DIR 202,1 4096 5120521 /edx/app/forum/cs_comments_service
tcp 0 0 127.0.0.1:8110 0.0.0.0:* LISTEN 25301/pythontcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 29652/nginx.conftcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 5540/epmdtcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 864/sshdtcp 0 0 0.0.0.0:18040 0.0.0.0:* LISTEN 29652/nginx.conftcp 0 0 0.0.0.0:15672 0.0.0.0:* LISTEN 5445/beam.smptcp 0 0 0.0.0.0:55672 0.0.0.0:* LISTEN 5445/beam.smptcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1092/mastertcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 29652/nginx.conftcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 19414/pythontcp 0 0 127.0.0.1:8130 0.0.0.0:* LISTEN 20880/pythontcp 0 0 0.0.0.0:8099 0.0.0.0:* LISTEN 29652/nginx.conftcp 0 0 127.0.0.1:8100 0.0.0.0:* LISTEN 22832/pythontcp 0 0 0.0.0.0:44805 0.0.0.0:* LISTEN 5445/beam.smptcp 0 0 127.0.0.1:8040 0.0.0.0:* LISTEN 28718/pythontcp 0 0 127.0.0.1:5672 0.0.0.0:* LISTEN 5445/beam.smptcp 0 0 127.0.0.1:4040 0.0.0.0:* LISTEN 1434/pythontcp 0 0 127.0.0.1:9001 0.0.0.0:* LISTEN 954/pythontcp 0 0 0.0.0.0:18090 0.0.0.0:* LISTEN 29652/nginx.conftcp 0 0 127.0.0.1:8010 0.0.0.0:* LISTEN 19057/pythontcp 0 0 127.0.0.1:9002 0.0.0.0:* LISTEN 960/pythontcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 12885/memcachedtcp6 0 0 :::9200 :::* LISTEN 1835/javatcp6 0 0 :::9300 :::* LISTEN 1835/javatcp6 0 0 :::22 :::* LISTEN 864/sshdtcp6 0 0 :::25 :::* LISTEN 1092/master
www-data 4885 13.2 1.9 315652 75484 ? Sl 02:35 0:02 ruby /edx/app/forum/cs_comments_service/bin/unicorn -c config/unicorn.rb
drwxrwxrwx 2 www-data www-data 4096 Apr 6 02:38 forum/
-rw-r--r-- 1 www-data www-data 5 Apr 6 02:39 forum/forum_unicorn.pid
FORUM_LISTEN_HOST: "0.0.0.0"FORUM_LISTEN_PORT: "4567"FORUM_USE_TCP: false
tcp 0 0 127.0.0.1:8110 0.0.0.0:* LISTEN 1486/pythontcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1003/nginx.conftcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 1293/epmdtcp 0 0 0.0.0.0:36595 0.0.0.0:* LISTEN 1325/beam.smptcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 867/sshdtcp 0 0 0.0.0.0:15672 0.0.0.0:* LISTEN 1325/beam.smptcp 0 0 0.0.0.0:55672 0.0.0.0:* LISTEN 1325/beam.smptcp 0 0 0.0.0.0:18040 0.0.0.0:* LISTEN 1003/nginx.conftcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1111/mastertcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 1003/nginx.conftcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 1464/pythontcp 0 0 127.0.0.1:8130 0.0.0.0:* LISTEN 1479/pythontcp 0 0 0.0.0.0:8099 0.0.0.0:* LISTEN 1003/nginx.conftcp 0 0 127.0.0.1:8100 0.0.0.0:* LISTEN 1458/pythontcp 0 0 127.0.0.1:8040 0.0.0.0:* LISTEN 1459/pythontcp 0 0 127.0.0.1:5672 0.0.0.0:* LISTEN 1325/beam.smptcp 0 0 127.0.0.1:4040 0.0.0.0:* LISTEN 1432/pythontcp 0 0 127.0.0.1:9001 0.0.0.0:* LISTEN 955/pythontcp 0 0 127.0.0.1:8010 0.0.0.0:* LISTEN 1465/pythontcp 0 0 127.0.0.1:9002 0.0.0.0:* LISTEN 961/pythontcp 0 0 0.0.0.0:18090 0.0.0.0:* LISTEN 1003/nginx.conftcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 992/memcachedtcp6 0 0 :::9200 :::* LISTEN 1850/javatcp6 0 0 :::9300 :::* LISTEN 1850/javatcp6 0 0 :::22 :::* LISTEN 867/sshdtcp6 0 0 :::25 :::* LISTEN 1111/master
, [2016-04-06T13:04:10.987099 #12831] INFO -- : Refreshing Gem list/edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/server_selector/selectable.rb:90:in `select_server': No server is available matching preference: #<Mongo::ServerSelector::Primary:0x000000051e8e80 @options={"mode"=>:primary, "database"=>:cs_comments_service_development, "write"=>{"w"=>1}, "read"=>{"mode"=>:primary}, "max_retries"=>1, "retry_interval"=>0, "timeout"=>0.5, "ssl"=>false}, @tag_sets=[], @server_selection_timeout=30> (Mongo::Error::NoServerAvailable) from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/collection/view/iterable.rb:40:in `block in each' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/retryable.rb:46:in `call' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/retryable.rb:46:in `read_with_retry' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/collection/view/iterable.rb:39:in `each' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongoid-5.0.0/lib/mongoid/query_cache.rb:207:in `each' from /edx/app/forum/cs_comments_service/app.rb:141:in `map' from /edx/app/forum/cs_comments_service/app.rb:141:in `<top (required)>' from config.ru:21:in `require' from config.ru:21:in `block in <main>' from /edx/app/forum/.gem/ruby/1.9.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `instance_eval' from /edx/app/forum/.gem/ruby/1.9.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `initialize' from config.ru:1:in `new' from config.ru:1:in `<main>' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/lib/unicorn.rb:48:in `eval' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/lib/unicorn.rb:48:in `block in builder' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:768:in `call' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:768:in `build_app!' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:137:in `start' from /edx/app/forum/.gem/ruby/1.9.1/gems/unicorn-4.9.0/bin/unicorn:126:in `<top (required)>' from /edx/app/forum/cs_comments_service/bin/unicorn:16:in `load' from /edx/app/forum/cs_comments_service/bin/unicorn:16:in `<main>'I, [2016-04-06T13:04:44.759549 #12967] INFO -- : Refreshing Gem list
> show databasesadmin 0.078GBedxapp 0.078GBlocal 0.078GB>
FORUM_MONGO_USER: "cs_comments_service"FORUM_MONGO_PASSWORD: "password"FORUM_MONGO_HOSTS: - "172.16.2.10"FORUM_MONGO_TAGS: !!nullFORUM_MONGO_PORT: "27017"FORUM_MONGO_DATABASE: "cs_comments_service"FORUM_MONGO_USE_SSL: false
mongo 172.16.2.10:27017/cs_comments_service -u cs_comments_service -p password
MongoDB shell version: 2.0.4cs_comments_service
>show dbs
admin 0.078125GBedxapp 0.078125GBlocal 0.078125GB> show users> show collections
`select_server': No server is available matching preference: #<Mongo::ServerSelector::Primary:0x0000000485d348 @options={"mode"=>:primary, "database"=>:cs_comments_service_development, "write"=>{"w"=>1}, "read"=>{"mode"=>:primary}, "max_retries"=>1, "retry_interval"=>0, "timeout"=>0.5, "ssl"=>false}, @tag_sets=[], @server_selection_timeout=30> (Mongo::Error::NoServerAvailable)
from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/collection/view/iterable.rb:40:in `block in each' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/retryable.rb:46:in `call' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/retryable.rb:46:in `read_with_retry' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongo-2.1.1/lib/mongo/collection/view/iterable.rb:39:in `each' from /edx/app/forum/.gem/ruby/1.9.1/gems/mongoid-5.0.0/lib/mongoid/query_cache.rb:207:in `each' from /edx/app/forum/cs_comments_service/app.rb:141:in `map' from /edx/app/forum/cs_comments_service/app.rb:141:in `<top (required)>' from config.ru:21:in `require' from config.ru:21:in `block in <main>' from /edx/app/forum/.gem/ruby/1.9.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `instance_eval' from /edx/app/forum/.gem/ruby/1.9.1/gems/rack-1.6.4/lib/rack/builder.rb:55:in `initialize' from config.ru:1:in `new' from config.ru:1:in `<main>' from /edx/app/forum/.gem/ruby/1.9.1/gems/
--
You received this message because you are subscribed to a topic in the Google Groups "Open edX operations" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openedx-ops/lFL-e0YVNbk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openedx-ops...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/9e226738-73e6-4c57-a2ce-68ed4a20438d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
user_info = user.to_dict()
File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/models.py", line 60, in to_dict
self.retrieve()
File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/models.py", line 65, in retrieve
self._retrieve(*args, **kwargs)
File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/user.py", line 151, in _retrieve
metric_tags=self._metric_tags,
File "/edx/app/edxapp/edx-platform/lms/lib/comment_client/utils.py", line 117, in perform_request
content=response.text[:100]
CommentClientError: u'Comments service returned invalid JSON for request bf285921-4913-4059-939f-3df09ef57a95; first 100 characters: \'<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor="white">\r\n<center><h1>502 Bad Gat\''
Oh, that's great Juanito!