IP range blocking with nginx-lua.

[Makailol Charls]

Hello,

I would like to add some blacklist and white list IPs to Nginx dynamically. I found simple example of blocking single IP with access_by_lua in lua-nginx-module document ( https://github.com/openresty/lua-nginx-module#synopsis ). but could not find the way to block the entire range of IPs.

Could someone explain how to block/whitelist range of IPs (subnet) or even entire country IPs using lua-nginx-module module?

Thanks,
Makailol

[Yichun Zhang (agentzh)]

Hello!

You can probably find an existing good enough Lua library doing this
on the web. Otherwise you can always code up a Lua module yourself.

Regards,
-agentzh

[Lord Nynex]

Hello,

I'd like to first point out that this functionality is supported by nginx out of the box and does not necessarily require Lua or Openresty. You can find relevant documentation on the access module at http://nginx.org/en/docs/http/ngx_http_access_module.html. The obvious downside here is maintaining a txt file but nginx is very good at understanding CIDR notations.

An example:

nginx.conf:

server {

  listen 80;

  include /etc/nginx/blacklist.txt

  ...

}

blacklist.txt:

deny 192.168.1.1/24

And more in line with your question about doing this dynamically. I've gone through a similar situation using a redis backed store (although the same is possible using shared dictionaries https://github.com/openresty/lua-nginx-module/#ngxshareddict). 

The biggest complication you will have is writing some utility functions to ensure the real ip address is within a range of addresses. In my opinion, this approach can be bad if the design is not well thought out. If you store an in memory blacklist that is in CIDR notation, it is likely that you will have to 'dump' the entire cache and match each address as a string. This does not yield the best performance. 

There are options, however. You can attempt to make a 'best guess' at the size of the subnet for the host.

For example, 

Given the IP address 192.168.1.10, we can (if we chose) assume that the most logical subnet for this IP is a /24 (192.168.1.0-192.168.1.255). 

If we use the above information we can arrive at the conclusion that the 'subnet host' is 192.168.1.0 (or 11000000 10101000 00000001 00000000 as a potential cache key) and this is consistent for the entire /24.

Given that assumption all requests that share this common subnet host should be denied. 

All of the above makes some pretty serious assumptions about your traffic and your app. It is worth exploring other possibilities as well. 

Here is a link to an unrelated project written in Lua that is capable of parsing/generating CIDR notation. IMHO it is a useful example of bit math in Lua. 

http://luci.subsignal.org/trac/browser/luci/trunk/libs/core/luasrc/ip.lua?rev=5194

I've written an example perl script that was written to help people understand CIDR addressing. It may or may not be useful to you:

https://github.com/lordnynex/perl-ascii-reports/blob/master/example

--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mex]

2014-07-15 16:51 GMT+02:00 Makailol Charls <maka...@gmail.com>:
> Hello,
>
> I would like to add some blacklist

> and white list IPs to Nginx.

if it is a static list that could be exported to
a textfile, what lord nyex said; woukd be nice to
have such a cponstruct perf-tested when having thousands
of ips/ranges in such a file.

> even entire country IPs using lua-nginx-module module?
>

https://github.com/agladysh/lua-geoip

you should, if possible, load the entire geoip-database on
starttime into cache; we had some performance-issues
when using geoip from python.

i'd like to ask you for some codesamples, if finished and successful :)


regards,


mex

[ish...@umbc.edu]

Using the "include /etc/nginx/blacklist.txt;" approach, one would have to reload the nginx config every time a change is made to this blacklist file, no? Assuming I wanted to do this without requiring a config reload, would the best way of handling this dynamically be to use access_by_lua and then use io.open to read and parse through the blacklist file and see if the IP is in it? Would the performance of this be nearly the same? Also in my case every IP will be /32, no need for subnet handling.

I'll probably end up using a Redis set and SMEMBERS to retrieve the IP list once I improve my application, but I'm curious about the file approach.

[Hamish Forbes]

If they are are just /32s and you don't need to do any subnet processing then a Lua table that looks something like this:

{
 ["123.123.123.123"] = true,
 ["10.10.10.10"] = true,
}

Should be pretty quick, you would just have to lookup use ngx.var.remote_addr as the key, if it returns nil then its not in your blacklist.

You could also use a background function using ngx.timer.at to re-populate the table periodically from a file without having to HUP or make network calls at request time.

[António P. P. Almeida]

Like others have suggested Redis is a good approach. If you want something simpler try CDB

http://asbradbury.org/projects/lua-tinycdb.

Just an idea.

----appa

[mex]

2014-07-16 14:49 GMT+02:00 <ish...@umbc.edu>:
> Using the "include /etc/nginx/blacklist.txt;" approach, one would have to
> reload the nginx config every time a change is made to this blacklist file,
> no?

yep. it really depends on how dynamic your file is and
how many entries. if it just changes once in a while
and is populated with a couple of entries i'd use the
file-approach

if your list is highly dynamic and fed from, lets say a waf or ids,
the list could grow fast, and parsing/chekcing could be a serious
performance-problem in the long run.
for this approach i would use redis/memcache or any other simple and
fast memorystore to store the ips/netranges.


regards,


mex

[Artur D]

Anyone implemented this without having to reload nginx ? 

I'm comfortable with any solution that wont involve reloading nginx 

[prkhr]

Artur,

You can use the ngx.timer to reload config stored in redis to shared dict in nginx at regular interval. 

[Artur D]

Thanks for the reply,

care to elaborate a bit ?

--
You received this message because you are subscribed to a topic in the Google Groups "openresty-en" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openresty-en/G916sAMSOPY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openresty-en...@googlegroups.com.

[prkhr]

Artur,

You can 

i) initialize a shared dictionary lua_shared_dict (https://github.com/openresty/lua-nginx-module#ngxshareddict)

ii)  Get it initialized in the init phase through ngx.timer.at (https://github.com/openresty/lua-nginx-module#ngxtimerat) in the init_worker_by_lua directive. The function passed to ngx.timer.at will be responsible for loading data from redis to shared dict. This func will also be responsible for setting the timer to run after n number of seconds.

iii) Use the shared dict get to find if the ip address exist in the dictionary

[Robert Paprocki]

Can we stop resurrecting a years-old thread? 😁

On Jun 19, 2018, at 19:01, Rob Kay <ark...@gmail.com> wrote:

I used https://github.com/hamishforbes/lua-resty-iputils and haven't had any problems 

--

You received this message because you are subscribed to the Google Groups "openresty-en" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.