Strange folders in /var/ngx_pagespeed_cache/

[JACK LINKERS]

Hello,

Beside the folders related to domains I'm hosting on my webserver, I see :

24x7-allrequestsallowed.com

85.17.141.182

baidu.com

httpheader.net

mafengwo.com

nixipdb.com

ntweb.org

prop_page

qunar.com

rname

url.cn

... some opf them are empty some are full of logs and cached files.

When I google I find many other websites I can access log files on with same folders.

Could anyone clarify from where these folders come from ?

Thanks

[Rainer Maurer]

i mean thats comes from referer spam

[Jeff Kaufman]

The prop_page and rname directories store pagespeed's property and metadata caches. 

All the other directories are the http cache.  Normally you'll only have ones for your site, but if someone gives you an invalid host header you can generate entries for other host names.

For example, I just checked ngxpagespeed.com and saw that the cache directory contained:

    

* !clean!time! (when was the cache last cleaned?)

* ngxpagespeed.com (http cache for ngxpagespeed.com)

* prop_page (property cache)
* rname (metadata cache)

Then I ran:

$ curl -D- -H 'Host: example.com' 'ngxpagespeed.com/ngx_pagespeed_example/rewrite_css_images.html

This curl command requests the page with the host header 'Host: example.com' instead of the expected 'Host: ngxpagespeed.com', which makes PageSpeed think that it's rewriting pages on example.com instead.  When I listed the contents of the cache directory again I saw that example.com had been added.

This is normally harmless, but if you want to avoid it you can lock down your server by specifying server names for all your virtual hosts and then adding a catchall block that gives 403-forbidden to everyone.  For example, I just set ngxpagespeed.com to have:

   server {

        listen 80;

        location / {

           deny all;

        }

   }

   server {

        listen       80;

        server_name  ngxpagespeed.com www.ngxpagespeed.com;

        pagespeed on;

        ...

    }

And now it's no longer possible to make it generate cache entries by supplying it with an unexpected Host header.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/5e2144fe-429d-4a32-9da0-8f6d18a14213%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Markus W.]

Hi @ all and thanks for spaceship Pagespeed - it's the pure incredible power of Google and just amazing.
I'm no expert in nginx but I installed Nginx 1.13.2 and compiled it successfully with ngx_pagespeed 1.12.34.2-stable + ngx_brotli
The reason why take this releases/versions are that only these versions were possible on my 1and1 Ubuntu 16.04.3 with Plesk 17.5.3 #20.
First I tried 1.12.34.4 but failed by #1451 (github) and after I solved it with the great patch by Otto van der Schaaf. But then came bug #1453. Therefor was no solution two weeks ago, too new. The little older version 1.12.34.2-stable was tested by one helpful Plesk team member (UFHH01) together with nginx 1.1.3.2, so I successfully installed Pagespeed for Nginx by Source (openssl-1-0-2l) by his tutorial two weeks ago.
Then I realized that not only my standard IPv4 and 2 additional Ipv4 directories plus my 8 domains directories were created in /var/ngx_pagespeed_cache/v3/. There are also 2 unknown domains from France and Great Britain I've never seen and heard about.
Today I was much more surprised that one of these domains changed to another unknown domain which seems to be a spam domain).
Otto recommended to describe the problem here. I can give server access on my own risk/responsibility if needed.

First two unknown domains

The yesterday evening changed domain (2nd one)









Hope I can help somewhere by describing this. More information possible (e.g. logs, access whatever), ask me.
Lots of greets

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.

[Otto van der Schaaf]

If you consider the entries in the cache directory a problem, you have two options to avoid them:

- restrict the hostnames that the vhost responds to in nginx - this probably does not have any drawbacks: Jeff posted an example of how to do so above.

- or you could turn off in-place resource optimization - the drawback is that optimization opportunities may be lost: https://www.modpagespeed.com/doc/system#ipro

Otto

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/5e2144fe-429d-4a32-9da0-8f6d18a14213%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/8928f979-1503-444f-813a-70b0563b1584%40googlegroups.com.

[sub scriibe]

Hi I would like to implement Jeff's suggestion to deny all unexcepted host headers.

I am on apache 2.4. I've added the following to the top of this file /etc/apache2/sites-enabled/000-default-le-ssl.conf

<VirtualHost *:80>

    ServerName catchall
    <Location />
        Require all denied
    </Location>

    <Location /var/www/html/>
        Require all denied
    </Location>
</VirtualHost>

When I try 

wget --header="Host: example.com" http://demo.mywebsite.com

Result -

Resolving demo.mywebsite.com (demo.mywebsite.com)... 142.41.74.25
Connecting to demo.mywebsite.com (demo.mywebsite.com)|142.41.74.25|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://demo.mywebsite.com/ [following]
--2022-09-05 10:57:46--  https://demo.mywebsite.com/
Reusing existing connection to demo.mywebsite.com:443.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://demo.mywebsite.com/ [following]
20 redirections exceeded.

Why is the output is 301 and not 403 Forbidden? 

When I refresh the cache folder example.com was added.

Does anyone know why its not working?

Here's the whole conf file -

<VirtualHost *:80>
    ServerName catchall
    <Location />
        Require all denied
    </Location>

    <Location /var/www/html/>
        Require all denied
    </Location>
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerAdmin webmaster@localhost

        ServerName demo.mywebsite.com
        ServerAlias demo.mywebsite.com

        Protocols h2 http/1.1

        DocumentRoot /var/www/html

        UseCanonicalName On
        UseCanonicalPhysicalPort On

        <Directory /var/www/html/>
            Options FollowSymLinks
            AllowOverride All
            Require all granted
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

SSLCertificateFile /etc/letsencrypt/live/demo.mywebsite.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/demo.mywebsite.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

Thank you if anyone can help.

[Borhan Uddin]

https://www.linuxhintbd.xyz/