python 3.5 / mod_wsgi / apache 2.4 / SOME static files are not found, others are

191 views
Skip to first unread message

joskra...@gmail.com

unread,
Jun 10, 2017, 12:56:59 PM6/10/17
to Mezzanine Users
Hello!

I have installed Mezzanine under Apache 2.4 and Python 3.5.  Most everything works great, except for one thing: certain static files are not found.  This problem does not occur if I start a project under Python 3.5 and use the runserver command- if I do that EVERYTHING works fine.

99% of the static files are found properly.  However... if I go to browse a gallery, the thumbnails are missing.  Also, if I do a blog post and insert a graphic from the gallery, the graphic is missing from the blog post.  If I inspect the HTML in the browser, for the missing images I have something like this:

<div class="col-xs-4 col-sm-3">
    <a class="thumbnail" rel="#image-1" title="Amsterdam  Holland" href="/static/media/uploads/gallery/Amsterdam%2C%20Holland.jpg">
        <img class="img-responsive" src="/static/media/">
    </a>
</div>

I should have something like this:

<div class="col-xs-4 col-sm-3"> <a class="thumbnail" rel="#image-1" title="Amsterdam Holland" href="/static/media/uploads/gallery/Amsterdam%2C%20Holland.jpg"> <img class="img-responsive" src="/static/media/uploads/gallery/.thumbnails/Amsterdam%2C%20Holland.jpg/Amsterdam%2C%20Holland-131x75.jpg"> </a> </div>

I have made sure that the security on the files in question is permissive, like so:

<Directory /srv/www/blog-sites/mywebsite/static>
Require all granted
</Directory>

<Directory /srv/www/blog-sites/mywebsite>
<Files wsgi.py>
Require all granted
</Files>
</Directory>


SELinux is disabled, at least for this testing. The only clue I can find is in the error logs:


[Sat Jun 10 12:52:44.250023 2017] [autoindex:error] [pid 4638] [client 140.254.70.225:34634] AH01276: Cannot serve directory /srv/www/blog-sites/mysite_net/static/media/: No matching DirectoryIndex (index.html) found, and server-generated directory index forbidden by Options directive, referer: http://www.mysite.net/test-post-two/

Does anyone have an idea as to what may be going wrong here?

Thanks!
-JK

joskra...@gmail.com

unread,
Jun 10, 2017, 1:04:55 PM6/10/17
to Mezzanine Users
One update.  I was able to get rid of the error related to Indexes, but the graphic URLs are still not being formed properly (and therefore, no graphics are shown).

Karim

unread,
Jun 10, 2017, 5:13:41 PM6/10/17
to Mezzanine ML

On Sun, Jun 11, 2017 at 3:04 AM, <joskra...@gmail.com> wrote:
One update.  I was able to get rid of the error related to Indexes, but the graphic URLs are still not being formed properly (and therefore, no graphics are shown).

​I have the same error as I described on my mail here in the list: "Can upload the media file, but no thumbnail e no right url in the posts".

I use python 3.6.1 on apache and the static files are managed by a different nginx instance. 

​The problem is the same. On django-filebrowser I don't get the thumbnails, but Pillow is correctly installed. I notice a 403 error after the upload when pointing `static/media`.

On the published post the src is not formed correctly but just point to the media folder even if on the rich text editor the image was correctly handled and pointed to the correct url and rendered.

How you solved this?

One update.  I was able to get rid of the error related to Indexes

​What I did so far:
- Checked and changed the file permissions on the server. 
​- Reinstalled Pillow
​- Checked the MEDIA* and STATIC* settings on production

At the moment still now working and I have no idea what to do to fix this.

​Cheers​

--
Karim N. Gorjux

Karim

unread,
Jun 10, 2017, 6:27:18 PM6/10/17
to Mezzanine ML
UPDATE

if STATIC_URL and MEDIA_URL are set with the `http://....` the img are correctly rendered in the posts with the right `src`.

The thumbnails generation in django-filebrowser is still broken because point to `static/media`


--
Karim N. Gorjux

Joshua Kramer

unread,
Jun 10, 2017, 6:39:47 PM6/10/17
to mezzani...@googlegroups.com
Karim, here is something else I found. In my case, there is a problem
importing PIL. On line 298 of the file:
/srv/www/blog-sites/lib/python3.5/site-packages/mezzanine/core/templatetags/mezzanine_tags.py

...it tries to import PIL, but it fails. When I switch to this
virtualenv and try to import Image from PIL, I get an error about
loading a JPEG shared library. So I need to chase down why this JPEG
library isn't there.
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Mezzanine Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/mezzanine-users/2_hf9eIc4eM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> mezzanine-use...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Karim

unread,
Jun 10, 2017, 6:44:42 PM6/10/17
to Mezzanine ML

On Sun, Jun 11, 2017 at 8:39 AM, Joshua Kramer <joskra...@gmail.com> wrote:
...it tries to import PIL, but it fails.  When I switch to this
virtualenv and try to import Image from PIL, I get an error about
loading a JPEG shared library.  So I need to chase down why this JPEG
library isn't there.

​check if you have all the requirements to compile Pillow correctly.


​If you check the requirements, you must have `libjpeg` installed and I suppose also `libjpeg-dev`.

​Have a look​


--
Karim N. Gorjux

Joshua Kramer

unread,
Jun 10, 2017, 11:14:47 PM6/10/17
to mezzani...@googlegroups.com
Ok, so there were two final issues. First, there were the three JPEG
libraries that were not installed on my target system. So I installed
them. (I am not sure why the Mezzanine site worked in debug mod even
without having those libraries installed, but it works now.)
Secondly, SELinux was denying access to httpd to execute those shared
libraries, even when they were there. So I used the SELinux
diagnostic tools to create SELinux policies to allow that to occur.
presto! Everything works now.

ProTip: There are some SELinux denials that are not logged by default!
A program that tries to execute a bit of code within a shared library
will silently fail if SELinux doesn't approve of that execution. The
documentation for RedHat 7 describes how to fix this. (Basically
rebuild the selinux policy with an option enabled, and reboot the
system. The problem is your logs will fill up quickly and you'll have
to sift through some stuff to find the denial. But then you can
rebuild the selinux policy without this option enabled to return to
normal operation. Once you sift through the denial you can use the
normal audit2allow and friends to create the correct policy.)

If anyone is curious: I am running this on CentOS 7 with the Software
Collections versions of Python 3.5 and Apache 2.4. Lots of custom
configuration here. But now it appears to work fully!

Melvyn Sopacua

unread,
Jun 11, 2017, 8:43:42 AM6/11/17
to mezzani...@googlegroups.com

Hi Karim,

If STATIC_URL fixes part of your problem, then your setting is incorrect. Make sure they are absolute, not relative, so /static/ not static/.

 

If no thumbnails are generated on disk, then (like Joshua) the problem is with Pillow.

 

You can test what happens in a django shell as follows:

>>> from mezzanine.core.templatetags.mezzanine_tags import thumbnail

>>> thumbnail('/media/uploads/foo.jpg', 80, 80)

 

(needs to be an existing image bigger then 80x80 for any Pillow runtime related errors to surface.)

--

Melvyn Sopacua

Karim

unread,
Jun 12, 2017, 4:13:34 AM6/12/17
to Mezzanine ML
​Thanks, I think you pointed me to the right way to solve this.

SuspiciousFileOperation: The joined path (.../kng_static/media/uploads/qa_fields.png) is located outside of the base path component (.../kng/kng_site/static/media)​

​This happens because the `static` dir in my project is a link to another dir.​

​Thanks​


--
Karim N. Gorjux

Melvyn Sopacua

unread,
Jun 12, 2017, 5:25:53 AM6/12/17
to mezzani...@googlegroups.com

On Monday 12 June 2017 18:12:58 Karim wrote:

> On Sun, Jun 11, 2017 at 10:43 PM, Melvyn Sopacua

> <m.r.s...@gmail.com>

> wrote:

> > On Sunday 11 June 2017 08:26:44 Karim wrote:

> > > UPDATE

> > >

> > >

> > >

> > > if STATIC_URL and MEDIA_URL are set with the `http://....` the img

> > > are

> > >

> > > correctly rendered in the posts with the right `src`.

> > >

> > >

> > >

> > > The thumbnails generation in django-filebrowser is still broken

> > >

> > > because point to `static/media`

> >

> > If STATIC_URL fixes part of your problem, then your setting is

> > incorrect. Make sure they are absolute, not relative, so /static/

> > not static/.

> >

> >

> >

> > If no thumbnails are generated on disk, then (like Joshua) the

> > problem is with Pillow.

> >

> > You can test what happens in a django shell as follows:

> > >>> from mezzanine.core.templatetags.mezzanine_tags import thumbnail

> > >>>

> > >>> thumbnail('/media/uploads/foo.jpg', 80, 80)

> >

> > (needs to be an existing image bigger then 80x80 for any Pillow

> > runtime related errors to surface.)

>

> ​Thanks, I think you pointed me to the right way to solve this.

>

> SuspiciousFileOperation: The joined path

> (.../kng_static/media/uploads/qa_fields.png) is located outside of the

> base path component (.../kng/kng_site/static/media)​

>

> ​This happens because the `static` dir in my project is a link to

> another dir.â€

 

Use os.path.realpath() in your settings file. The drawback being that changing the target of the symlink requires a reload of Django/WSGI for it to be picked up. This is not by definition a bad thing.

 

A similar approach is to implement your own storage engine and call os.path.realpath on STATIC_ROOT. Of course, this opens you up to symlink attacks.

 

Personally, I use bind mounts.

--

Melvyn Sopacua

Karim

unread,
Jun 12, 2017, 6:48:09 AM6/12/17
to Mezzanine ML

On Mon, Jun 12, 2017 at 7:25 PM, Melvyn Sopacua <m.r.s...@gmail.com> wrote:

 

Use os.path.realpath() in your settings file. The drawback being that changing the target of the symlink requires a reload of Django/WSGI for it to be picked up. This is not by definition a bad thing.

 

A similar approach is to implement your own storage engine and call os.path.realpath on STATIC_ROOT. Of course, this opens you up to symlink attacks.


​I'm in a shared hosting (webfaction) so my hands are tied and I can't do what I want.

Fixed the MEDIA_ROOT and STATIC_ROOT. Also the urls are ok

In [4]: from django.conf import settings
In [5]: settings.STATIC_URL
Out[5]: '/static/'

In [6]: settings.STATIC_ROOT
Out[6]: '/home/
​myuser
/webapps/kng_static/'

In [7]: settings.MEDIA_URL
Out[7]: '/static/media/'

In [8]: settings.MEDIA_ROOT
Out[8]: '/home/
​myuser​
/webapps/kng_static/media/'


​If I try the mezzanine_tags:​

In [3]: from mezzanine.core.templatetags.mezzanine_tags import thumbnail
   ...: img = '/home/
​myuser
/webapps/kng_static/media/upload/
​test_image
.jpg'
   ...: thumbnail(img, 80, 80)
   ...:
Out[3]: '/home/systemx/
​myuser
/kng_static/media/upload/.thumbnails/
test_image
.jpg
​/​
test_image
-80x80.jpg'

​The problem is still the same in the filebrowser. No thumbnail because the src of the img pointed is `www.mywebsite.com/static/media/`and if I use the filebrowser on a post, the image rendered use the src `www.mywebsite.com/static/media/`

​The permissions are ok and I can see the images if I point to the specific url in the browser.

I think I must implement the logging and fight my laziness...​

​Thank you for your support Melvyn​


--
Karim N. Gorjux

Melvyn Sopacua

unread,
Jun 12, 2017, 10:00:04 AM6/12/17
to mezzani...@googlegroups.com

On Monday 12 June 2017 20:47:33 Karim wrote:

 

> ​If I try the mezzanine_tags:​

>

> In [3]: from mezzanine.core.templatetags.mezzanine_tags import

> thumbnail ...: img = '/home//myuser/webapps/kng_static/media/upload/test_image.jpg'

 

Thumbnail takes a URL not a filesystem path.

 

--

Melvyn Sopacua

Reply all
Reply to author
Forward
0 new messages