Can perlbal limit # of connections per IP?

[ichudov]

I have some websites serving large PDF files and sometimes I am hit by
mad "download accelerators". They make hundreds of simultaneous
requests for pages.

I want to thwart them in the nicest possible way (without
inconveniencing the user who, likely, has no clue).

Does perlbal offer something like this?

Alternatively, does mod_limitipconn work with perlbal (who originates
all connections from 127.0.0.1).

Thanks!

i

[Adam Thomason]

I wrote a plugin called Throttle to handle situations like that. You configure the number of allowed concurrent connections per IP, and it queues up new connections if the limit is exceed.

I thought we published it somewhere, but I can't find it in a public place now. I'll try to put together a proper release, but meanwhile see attached.

Adam

[ichudov]

Adam, this is very interesting. I looked at Throttle.pm and it appears
to limit the number of accesses per period of time, as opposed to the
number of simultaneous connections. So, a page with many embedded
images, would trip the limiter, even if all connections were made in
series.

``Perlbal plugin that throttles connections from hosts that connect
too frequently''

Perhaps I am missing something?

i

>  Throttle.pm
> 26KViewDownload
>
>  FastACL.pm
> 8KViewDownload

[Adam Thomason]

That's the first line of defense, but there's also a simultaneous connection limit:

        max_concurrent => {
            check_role => '*',
            des => "Maximum number of connections accepted at a time from a single IP, per perlbal instance.",
            default => 2,
            check_type => 'int',
        }

This limit applies to the connections in the waiting-to-read-headers state, though, not those which have been accepted. You'd have to add a counter for accepted connections per IP to make that work.

Adam

[ichudov]

Adam, thank you. I will look into this.

I have, however, found what looks like a perfect solution. I will see
if it works.

All I need to do to thwart those download accelerators is to tell them
that I do NOT accept byte ranges, by sending them a header:

Accept-Ranges: none

That can be easily done with mod_headers.

In addition, the PDF files are stored in a database and served by a
perl script (even though the URLs look like files on a filesystem).
So, I think that if I see a Range header, I may as well deny the
request with a 403 or something.

Since now I am sending Accept-Ranges: none for all my sites.

Thanks a lot.

i

[hachi]

Your solution does not prevent people from downloading multiple at the same time, and some PDF plugins may make use of Range requests to only download the byte range they need to render at the current time.

I don't know what your precise use case is, but you may actually find an anti-abuse tool like the throttler plugin will give you far better results in the long run.

--hachi

[Adam Thomason]

There's an improved version of the Throttle plugin here: https://github.com/athomason/Perlbal/blob/throttle/lib/Perlbal/Plugin/Throttle.pm. The dependencies are now optional (if you don't need some of the white/blacklisting, logging, and memcached state sharing features). The configuration and documentation are clearer also.
Adam