Skipping logging for known URLs

[Graeme Defty]

Evan,

My production site takes hundreds of hits a day from morons who runs scripts testing for what are presumably default admin URLs in some other, lesser frameworks, for example:

/
/admin.php
/admin/login.php

etc etc

Naturally, these don't do any harm, but it fills up the error log and makes it hard to spot any genuine problems. One solution would be to write something to filter the log, but a better approach is just to silently ignore them, as they deserve.

Is it possible to add somewhere a list against which such errors could be tested to see if logging can be skipped? I guess a list of regular expressions would be nice.

(Sorry to be lazy. I know I should dig into the code and send you the patch, but I am a bit pushed for time right now, and I know you had already determined how it should be done before you finished reading my request ;-) ).

Cheers,

g

[Evan Miller]

My first thought is to add a "blackhole" destination to the routes system; that way you will be able to take advantage of regular expressions once we add support for it. In the meantime you might define routes for each of these and target a controller that returns {output, "Nope"}.

My second thought is to punish the offenders by tying up the socket for a minute or so.... you might do this by adding to the controller

receive
after 2 * 60 * 1000 -> % 2 minutes
   {output, "Nope"}
end

It'll cost about 4-8KB in memory overhead for each pending request, but maybe they'll take you off the target list...

Anyway the routes system is documented here:

http://www.chicagoboss.org/projects/chicagoboss/wiki/Routes

---

Graeme Defty May 20, 2011 11:14 AM

[Graeme Defty]

I must say the evil side of me likes the second suggestion. (Actually the good-guy side thinks its pretty cool too.

Routes? Wow! Must have missed that one. (Still hanging back on the upgrade because of the compiler issue. :-(  )

Thanks for the suggestions.

g
_____________________________________

[Jose Luis Gordo Romero]

Hi Graeme,

Add support for regular expressions in the new router system should be easy, I will look at it next week. With this you can route your bad traffic and do what you want (ignore it, be bad, ...)

Best

--

Jose Luis Gordo Romero

@jgordor - http://www.freemindsystems.com

2011/5/20 Graeme Defty <graeme...@gmail.com>

[Andre Nathan]

On Fri, 2011-05-20 at 11:33 -0500, Evan Miller wrote:
> My second thought is to punish the offenders by tying up the socket
> for a minute or so....

Wouldn't that make boss vulnerable to a DoS attack where sending lots of
requests to routes that end up in the blackhole will make the system use
all its available file descriptors?

Andre

[Evan Miller]

Yes.

--
Evan Miller
http://www.evanmiller.org/

[Graeme Defty]

At the risk of being just a little pedantic (go on - indulge me - its been a tough week) it would be more correct to say it would make *my application* susceptible to DoS attacks (and a very good point it is too - thanks)

It seems to me that having regex capability in the routing mechanism is still a good thing. In and of itself it does not weaken CB. Like most things, it is how we use it that creates or does not create the problem.

g
____________________________________________

[Andre Nathan]

On Sat, 2011-05-21 at 08:06 +0700, Graeme Defty wrote:
> It seems to me that having regex capability in the routing mechanism
> is still a good thing. In and of itself it does not weaken CB. Like
> most things, it is how we use it that creates or does not create the
> problem.

Sure, regexes on routes would be great. I was referring to "tying up the
socket" part (and assuming it would be a default CB behavior to do this
automatically for blackholed requests; maybe I misunderstood the idea).

Best,
Andre

[Graeme Defty]

A,

Yes, Evan's suggestion originally was that I could cause the socket to be held for a while, rather than have it as CB's normal behaviour.

I still like the idea (Strike Back ! Strike Back !) but I am not sure that I will do it. I do not get enough hits for DoS to be a concern, but why open the risk?

I guess I will just do the sensible thing and drop these hits with the minimum of fuss.

Anyway, thanks for the input.

g
___________________________________________

[Graeme Defty]

I may finally get round to doing something about this, but . . . how do I stop the logging (which was my original problem)?

g

_______________________________