Getting "Server Error" trying to access ecommerce site

[Juanito]

Hi, I keep getting Server Error error on the ecommerce site, and when I check /edx/var/ecommerce/edx.log this is what I get:

Mar 15 17:10:13 ip-10-33-196-33 [service_variant=ecommerce][django.request][env:no_env] ERROR [ip-10-33-196-33  31761] [base.py:256] - Internal Server Error: /favicon.ico

Traceback (most recent call last):

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 108, in get_response

    response = middleware_method(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/middleware.py", line 10, in process_request

    request.site = get_current_site(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/shortcuts.py", line 15, in get_current_site

    return Site.objects.get_current(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 62, in get_current

    return self._get_site_by_request(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 46, in _get_site_by_request

    site = self.get(domain__iexact=host)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/db/models/query.py", line 334, in get

    self.model._meta.object_name

DoesNotExist: Site matching query does not exist.

the cms worker dies too 

analytics_api                    RUNNING   pid 1730, uptime 0:55:01

certs                            RUNNING   pid 8320, uptime 0:49:17

ecommerce                        RUNNING   pid 31755, uptime 0:55:25

ecomworker                       RUNNING   pid 32046, uptime 0:55:18

edx_notes_api                    RUNNING   pid 3910, uptime 0:50:46

edxapp:cms                       RUNNING   pid 30910, uptime 0:56:54

edxapp:lms                       RUNNING   pid 30924, uptime 0:56:53

edxapp_worker:cms_default_1      EXITED    Mar 15 04:55 PM

edxapp_worker:cms_high_1         EXITED    Mar 15 04:55 PM

edxapp_worker:cms_low_1          EXITED    Mar 15 04:55 PM

edxapp_worker:lms_default_1      RUNNING   pid 15655, uptime 0:20:06

edxapp_worker:lms_high_1         RUNNING   pid 15658, uptime 0:20:06

edxapp_worker:lms_high_mem_1     RUNNING   pid 15653, uptime 0:20:06

edxapp_worker:lms_low_1          RUNNING   pid 15657, uptime 0:20:06

forum                            RUNNING   pid 8886, uptime 0:48:38

insights                         RUNNING   pid 3545, uptime 0:50:54

notifier-celery-workers          RUNNING   pid 8853, uptime 0:48:49

notifier-scheduler               RUNNING   pid 8827, uptime 0:49:00

xqueue                           RUNNING   pid 7989, uptime 0:49:24

xqueue_consumer                  RUNNING   pid 8018, uptime 0:49:22

The ecommerce runs under https://ecommerce.mydomain.com:30002 

EDXAPP_ECOMMERCE_PUBLIC_URL_ROOT: "https://ecommerce.{{ AVA_DOMAIN }}:30002"

EDXAPP_ECOMMERCE_API_URL: "https://ecommerce.{{ AVA_DOMAIN }}:30002/api/v2"

ECOMMERCE_NGINX_PORT: 30001

ECOMMERCE_SSL_NGINX_PORT: 30002

ECOMMERCE_ECOMMERCE_URL_ROOT: 'https://ecommerce.{{ AVA_DOMAIN }}:30002'

ECOMMERCE_LMS_URL_ROOT: "https://{{ EDXAPP_SITE_NAME }}"
ECOMMERCE_WORKER_ECOMMERCE_API_ROOT: '{{ ECOMMERCE_ECOMMERCE_URL_ROOT }}/api/v2/'

Initially I was trying to run it under port 443, but I run into nginx errors because lms is using that port 443 (SSL)

Any tips would be appreciate it

Thanks

[Jacek Bzdak]

As far as I can tell, worker failure is entirely unrelated to ECommerce, it is a know issue tracked on ticked OPS-1367, issue is important and seems to be getting more traction, so should be fixed soon. 

The error you see is related to Django Sites framework, used by Oscar payment processor (that is backbone of ECommerce), maybe go to admin and in Ecommerce add a Site instance for this domain: 'https://ecommerce.{{ AVA_DOMAIN }}:30002' (note this is a guesswork on my side :). Please keep us posted with any issues you find (I'm especially interested as I'll be setting up ecommerce on a sandbox in coming days :). 

-- 

Jacek 

@OpenCraft

--
You received this message because you are subscribed to the Google Groups "Open edX operations" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openedx-ops...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/003fefb9-8c26-420f-ae81-cf67b16ef94d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ki Hwan Song]

Thanks for the reply, yea the worker dies when I hit the ecommerce URL. I didn't know where was an issue, do you have a link to it?

I did try to access the ecommerce admin site but it throws server error just the same. I did have it running one time until I decided to add SSL configuration, that's when everything went haywire.

Going to backtrack.

--
You received this message because you are subscribed to a topic in the Google Groups "Open edX operations" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openedx-ops/xFwCnpesR0Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openedx-ops...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/CAJ-81QpdM45%3DooOyWX3eK7mZEhAVvQ%2BJDtgzjU6_K0jzm1hq4w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Ki Hwan Song
송기환

[Jacek Bzdak]

Hi, 

we are having problems with workers, that die on provisioning this error is tracked here (however I have no idea if it's visible to the world): https://openedx.atlassian.net/browse/OPS-1367, good thing is that this particular error is fixed by this pr: https://github.com/edx/edx-platform/pull/11814 so just merge it and see if it works. 

If your workers die when you enter some url, this most probably is another issue, so please post revelant logs here for more in-depth troubleshooting :) 

To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/CAF47s4x_%2BtNmT8XfFzhrBxd8M%2B4348pXZnDtgzK%2Bt8_mBONExw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Jacek 

@OpenCraft

[Juanito]

I started installation from scratch again, installed ecommerce, ecommerce worker gives "Server Error" when I hit the URL.

/edx/var/log/ecommerce/edx.log

Mar 16 13:14:29 ip-10-166-187-233 [service_variant=ecommerce][django.request][env:no_env] ERROR [ip-10-166-187-233  6194] [base.py:256] - Internal Server Error: /favicon.ico

Traceback (most recent call last):

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 108, in get_response

    response = middleware_method(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/middleware.py", line 10, in process_request

    request.site = get_current_site(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/shortcuts.py", line 15, in get_current_site

    return Site.objects.get_current(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 62, in get_current

    return self._get_site_by_request(request)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 46, in _get_site_by_request

    site = self.get(domain__iexact=host)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/edx/app/ecommerce/venvs/ecommerce/local/lib/python2.7/site-packages/django/db/models/query.py", line 334, in get

    self.model._meta.object_name

DoesNotExist: Site matching query does not exist.

I checked /edx/var/log/ecomworker/edx.log and I found connection refused

Trying again in 32.00 seconds...

Mar 16 13:17:22 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  6453] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

Mar 16 13:17:54 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  6453] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

Mar 16 13:18:26 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  6453] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

Mar 16 13:18:58 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  6453] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

Mar 16 13:19:30 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  6453] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

This message keeps repeating

I think rabbitmq is running fine?

sudo rabbitmqctl status

Status of node 'rabbit@ip-10-166-187-233' ...

[{pid,26808},

 {running_applications,

     [{rabbitmq_management,"RabbitMQ Management Console","3.2.3"},

      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.2.3"},

      {rabbit,"RabbitMQ","3.2.3"},

      {mnesia,"MNESIA  CXC 138 12","4.5"},

      {os_mon,"CPO  CXC 138 46","2.2.7"},

      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.2.3"},

      {webmachine,"webmachine","1.10.3-rmq3.2.3-gite9359c7"},

      {mochiweb,"MochiMedia Web Server","2.7.0-rmq3.2.3-git680dba8"},

      {xmerl,"XML parser","1.2.10"},

      {inets,"INETS  CXC 138 49","5.7.1"},

      {amqp_client,"RabbitMQ AMQP Client","3.2.3"},

      {sasl,"SASL  CXC 138 11","2.1.10"},

      {stdlib,"ERTS  CXC 138 10","1.17.5"},

      {kernel,"ERTS  CXC 138 10","2.14.5"}]},

 {os,{unix,linux}},

 {erlang_version,

     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:30] [kernel-poll:true]\n"},

 {memory,

     [{total,37469696},

      {connection_procs,1944128},

      {queue_procs,692000},

      {plugins,436160},

      {other_proc,9463584},

      {mnesia,119928},

      {mgmt_db,2166072},

      {msg_index,52656},

      {other_ets,1148600},

      {binary,591608},

      {code,17376751},

      {atom,1643137},

      {other_system,1835072}]},

 {vm_memory_high_watermark,0.4},

 {vm_memory_limit,3030650060},

 {disk_free_limit,50000000},

 {disk_free,89299959808},

 {file_descriptors,

     [{total_limit,924},

      {total_used,18},

      {sockets_limit,829},

      {sockets_used,16}]},

 {processes,[{limit,1048576},{used,436}]},

 {run_queue,0},

 {uptime,31737}]

Anyone have any suggestions?

[Juanito]

Well, just an update. I started configuring my server and I specified the following variables:

AVA_DOMAIN: 'mydomain.com'

EDXAPP_SITE_NAME: 'localhost'

EDXAPP_LMS_SITE_NAME: "stage.{{ AVA_DOMAIN }}"

EDXAPP_CMS_SITE_NAME: "studio.{{ AVA_DOMAIN }}"

EDXAPP_CMS_BASE: "{{ EDXAPP_CMS_SITE_NAME }}"

EDXAPP_CMS_NGINX_PORT: 80

I ran the playbook again and I don't get the Server Error anymore but I get the Oscar website (screenshot attached)

All workers are still alive.  

When I click login it takes me to http://127.0.0.1:8000/oauth2/authorize/?nonce=4UtQBITltczSsGGVBaZCh2Yy8R8ZmQVmuq4eXjdHeva2LVKzudjXTlnKj5lDPc0o&state=9hj6UPnitxH2DdRJySKEjeXqiNNpfqjz&redirect_uri=http://stage.mydomain.com:18130/complete/edx-oidc/&response_type=code&client_id=ecommerce-key&scope=openid+profile+email+permissions

I'm assuming 127.0.0.1:8000 is the LMS.  I'll need to set the right variables

Well ecomworker still outputs

Mar 16 20:50:36 ip-10-166-187-233 [service_variant=ecomworker][celery.worker.consumer][env:no_env] ERROR [ip-10-166-187-233  8722] [consumer.py:364] - consumer: Cannot connect to amqp://celery:**@10.166.187.233:5672//: [Errno 111] Connection refused.

Trying again in 32.00 seconds...

I am painstakingly setting variables one by one 

[Jacek Bzdak]

Hi Juanito, 

I have successfully configured ECommerce on sandbox, and I also ran into issues with the Sites framework (albeit different). 

Django sites framework work by attaching a site object to every request object, this site is attached either by matching the requested domain, or using a SITE_ID setting from Django settings. SITE_ID has precedence over the other mechanism, and is set in Otto to SITE_ID=1, so for every request Django tries to find site with id equal to 1, which happens to be an ``example.com`` site that should be installed by default. 

If not following snippet should in ECommerce shell add it: 

from django.contrib.sites.models import Site; Site.objects.create(pk=1, name='example.com', domain='example.com')

-- 

Jacek 

@OpenCraft

To view this discussion on the web visit https://groups.google.com/d/msgid/openedx-ops/2f4f80b9-0af2-43e2-ab00-3bca48e79e9b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jacek 

@OpenCraft

[Juanito]

Yey, congrats on your progress.

Your tip really helped.

Still working on getting it work right.

Thanks