error when uploading photo zip archive

[jcu...@gmail.com]

Hello All,

I have been having this problem when trying to upload a zipfile containing 10 photos (64mb). 

Internal Server Error: /admin/photologue/photo/upload_zip/
KeyError at /admin/photologue/photo/upload_zip/
u'request'

Internal Server Error: /admin/photologue/photo/upload_zip/
MemoryError at /admin/photologue/photo/upload_zip/
No exception message supplied

This happens only on my production server with 4gb for ram. My test server handles the upload without issues.

This is what I am running in both environments:

Django 1.10.3

django-photologue 3.6

Pillow 3.4.2

python 2.7.12

Did anyone encounter a similar issue? I have been spending the last couple of hours trying to fix this and I am running out of options.

Any hint will be appreciated!

Julian

[Richard Barran]

It would help a lot to have a stacktrace - the error message below is very short!

Could you switch on DEBUG mode in the settings on your production server and re-run this test, and then post the output here?

[jcu...@gmail.com]

Hello Richard,

Thank you for the quick reply. 

Below is the stacktrace. But I have also noticed another issue, which may be related to this. When trying to run the management command plcache I get an error:

./manage.py plcache --help

Traceback (most recent call last):

  File "./manage.py", line 13, in <module>

    execute_from_command_line(sys.argv)

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line

    utility.execute()

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/management/__init__.py", line 208, in fetch_command

    klass = load_command_class(app_name, subcommand)

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/management/__init__.py", line 40, in load_command_class

    module = import_module('%s.management.commands.%s' % (app_name, name))

  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module

    __import__(name)

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/management/commands/plcache.py", line 7, in <module>

    class Command(BaseCommand):

  File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/management/commands/plcache.py", line 8, in Command

    option_list = BaseCommand.option_list + (

AttributeError: type object 'BaseCommand' has no attribute 'option_list'

Stacktrace here:

Internal Server Error: /admin/photologue/photo/upload_zip/

KeyError at /admin/photologue/photo/upload_zip/

u'request'

Request Method: POST

Django Version: 1.10.3

Python Executable: /home/pcs/.virtualenvs/pcsapps/bin/uwsgi

Python Version: 2.7.12

Python Path: ['.', '', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/pyrepl-0.8.4-py2.7.egg', '/home/pcs/.virtualenvs/pcsapps/lib/python27.zip', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/plat-linux2', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/lib-tk', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/lib-old', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages']

Server time: Fri, 11 Nov 2016 08:55:28 -0500

Installed Applications:

('django.contrib.admin',

'django.contrib.auth',

'django.contrib.contenttypes',

'django.contrib.sessions',

'django.contrib.sites',

'django.contrib.messages',

'django.contrib.staticfiles',

'django.contrib.humanize',

'django.contrib.admindocs',

'localflavor',

'ckeditor',

'ckeditor_uploader',

'compat',

'django_extensions',

'hijack',

'import_export',

'sitetree',

'taggit',

'mptt',

'django_select2',

'easy_thumbnails',

'photologue',

'sortedm2m',

'utphy_kerberos',

'events',

'kiosk',

'locations',

'mailman',

'people',

'phones',

'keys',

'utphy_photoalbums')

Installed Middleware:

['django.middleware.common.CommonMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',

'django.middleware.csrf.CsrfViewMiddleware',

'django.contrib.auth.middleware.AuthenticationMiddleware',

'django.contrib.auth.middleware.SessionAuthenticationMiddleware',

'django.contrib.messages.middleware.MessageMiddleware',

'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:  

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner

 39.             response = get_response(request)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/utils/deprecation.py" in __call__

 136.             response = self.get_response(request)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner

 41.             response = response_for_exception(request, exc)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/exception.py" in response_for_exception

 86.         response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/exception.py" in handle_uncaught_exception

 135.     return callback(request, **param_dict)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view

 149.                     response = view_func(request, *args, **kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/views/defaults.py" in server_error

 74.     return http.HttpResponseServerError(template.render())

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/backends/django.py" in render

 66.             return self.template.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render

 208.                     return self._render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in _render

 199.         return self.nodelist.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render

 994.                 bit = node.render_annotated(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render_annotated

 961.             return self.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/loader_tags.py" in render

 174.         return compiled_parent._render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in _render

 199.         return self.nodelist.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render

 994.                 bit = node.render_annotated(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render_annotated

 961.             return self.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/loader_tags.py" in render

 70.                 result = block.nodelist.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render

 994.                 bit = node.render_annotated(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render_annotated

 961.             return self.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/loader_tags.py" in render

 70.                 result = block.nodelist.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render

 994.                 bit = node.render_annotated(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/base.py" in render_annotated

 961.             return self.render(context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/sitetree/templatetags/sitetree.py" in render

 203.         tree_items = sitetree.menu(self.tree_alias, self.tree_branches, context)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/sitetree/sitetreeapp.py" in menu

 732.             if not item.hidden and item.inmenu and self.check_access(item, context):

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/sitetree/sitetreeapp.py" in check_access

 758.         authenticated = self._global_context['request'].user.is_authenticated()

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/template/context.py" in __getitem__

 75.         raise KeyError(key)

Exception Type: KeyError at /admin/photologue/photo/upload_zip/

Exception Value: u'request'

Thank yo for looking into this!

Julian

[m1chael]

Hello,

Do you have 'django.template.context_processors.request' added appropriately in your settings?

https://docs.djangoproject.com/en/1.10/ref/templates/api/#using-requestcontext

This may be your culprit

Mike

--
You received this message because you are subscribed to the Google Groups "Django Photologue" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-photologue+unsubscribe@googlegroups.com.
To post to this group, send email to django-photologue@googlegroups.com.
Visit this group at https://groups.google.com/group/django-photologue.
For more options, visit https://groups.google.com/d/optout.

[jcu...@gmail.com]

Hello Mike,

Yes,  'django.template.context_processors.request' is in the list of context_processors.

Thanks for checking!

Julian

To unsubscribe from this group and stop receiving emails from it, send an email to django-photolo...@googlegroups.com.
To post to this group, send email to django-p...@googlegroups.com.

[jcu...@gmail.com]

Hi Richard,

one more traceback below. This time it seems to complain about /site-packages/PIL/ImageFile.py in load_prepare, line 259

Environment:

Request Method: POST

Django Version: 1.10.3

Python Version: 2.7.12

 'utphy_photoalbums',

 'debug_toolbar')

Installed Middleware:

['django.middleware.common.CommonMiddleware',

 'django.contrib.sessions.middleware.SessionMiddleware',

 'django.middleware.csrf.CsrfViewMiddleware',

 'django.contrib.auth.middleware.AuthenticationMiddleware',

 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',

 'django.contrib.messages.middleware.MessageMiddleware',

 'django.middleware.clickjacking.XFrameOptionsMiddleware',

 'debug_toolbar.middleware.DebugToolbarMiddleware']

Traceback:

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner

  39.             response = get_response(request)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response

  187.                 response = self.process_exception_by_middleware(e, request)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response

  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view

  149.                     response = view_func(request, *args, **kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func

  57.         response = view_func(request, *args, **kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner

  211.             return view(request, *args, **kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/admin.py" in upload_zip

  215.                 form.save(request=request)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/forms.py" in save

  169.             photo.image.save(filename, contentfile)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/django/db/models/fields/files.py" in save

  97.             self.instance.save()

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/models.py" in save

  555.         super(Photo, self).save(*args, **kwargs)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/models.py" in save

  506.         self.pre_cache()

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/models.py" in pre_cache

  474.                 self.create_size(photosize)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/models.py" in create_size

  433.             im = self.resize_image(im, photosize)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/photologue/models.py" in resize_image

  409.             im = im.resize(new_dimensions, Image.ANTIALIAS)

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/PIL/Image.py" in resize

  1544.         self.load()

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/PIL/ImageFile.py" in load

  179.         self.load_prepare()

File "/home/pcs/.virtualenvs/pcsapps/lib/python2.7/site-packages/PIL/ImageFile.py" in load_prepare

  259.             self.im = Image.core.new(self.mode, self.size)

Exception Type: MemoryError at /admin/photologue/photo/upload_zip/

Exception Value: 

[Richard Barran]

Hi,

Thanks for the stack trace - it’s very informative.

First issue: the command plcache seems to be buggy - I will investigate and fix this.

Second issue: the "Exception Value: u’request’”.

This seems to be triggered in the application ‘django-sitetree’, which I am not familiar with. Sorry, I won’t be able to help much with this issue!

Richard

[jcu...@gmail.com]

Hello Richard,

Thank you for looking into this! 

I will have to fix the issue one way or another! I will report the fix (if I find any) in this thread!

Julian 

[jcu...@gmail.com]

Hello Richard,

I was able to solve the problem. It turns out that it is not directly related to photologue, but to the size of the zip file I was using.

The uWsgi config script was having 2 restrictions: 'harakiri' and 'limit_as' which were blocking the process. 

I have increased the values and now everything is working as expected.

I hope this will prove helpful for someone else.

Julian

On Saturday, November 12, 2016 at 3:12:32 PM UTC-5, Richard Barran wrote:

[Richard Barran]

I’m glad that you fixed the issue!