Debian 12 - IPv4 blocked without fail2ban & co

Romain

unread,

Sep 6, 2023, 3:00:07 AM9/6/23

to

Hello everyone,

I recently reinstalled a dedicated server (with OVH, in case that helps with diagnosis), transitioning from Debian 11 to Debian 12.

Since then, I have been experiencing regular and progressive IPv4 blocking at my home. Access via IPv6 or from another IPv4 (including from the same provider) is working.

Here's an example from last night after a server reboot the previous evening:
01:43-02:13 (30 minutes)
02:26-03:26 (1 hour)
03:28-05:28 (2 hours)
05:55-? (still blocked at the time of writing)

Issues:
- The OVH Debian 12 template for dedicated servers doesn't come with any pre-installed blocking tools (e.g., no fail2ban).
- I haven't added any such tools since installing the server, only Apache (with mod_security), PHP, and MariaDB.
- From my home IP address last night, the only generated requests were from my Uptime Kuma probe, which includes a ping every minute and a curl request to a URL that consistently returns HTTP 200 (except when the IP is blocked, of course).

When my IP is blocked, curl returns a "Connection refused," and ping returns "Destination Port Unreachable."

I couldn't find any mentions of my IPv4 address in the server logs. MTR (-4) doesn't report any issues reaching the server. OVH has confirmed that it's not an issue with their network equipment, and a rescue mode restart allows me to regain ping access.

Do you have any ideas about what might be causing this on a nearly pristine Debian 12 installation? I've been pulling my hair out for a few days now...

Thanks!

Romain

Michel Verdier

unread,

Sep 6, 2023, 4:50:06 AM9/6/23

to

On 2023-09-06, Romain wrote:

> I couldn't find any mentions of my IPv4 address in the server logs. MTR
> (-4) doesn't report any issues reaching the server.

Did you run mtr from the both sides ?

Romain

unread,

Sep 6, 2023, 4:50:06 AM9/6/23

to

No but's this is what I plan to do next time :)

Andy Smith

unread,

Sep 6, 2023, 5:00:06 AM9/6/23

to

Hi,

On Wed, Sep 06, 2023 at 08:39:55AM +0200, Romain wrote:
> When my IP is blocked, curl returns a "Connection refused," and ping
> returns "Destination Port Unreachable."
>
> I couldn't find any mentions of my IPv4 address in the server logs. MTR
> (-4) doesn't report any issues reaching the server.

So when this is happening mtr works but http, ssh and ping don't?

If you run tcpdump on the server, do your http/ssh packets even
reach it?

If you do tcptraceroute on port 22/80/443 from your home, how far
does it get?

Are there any firewall rules in place on your server?

DO you have access to any other host so you can tell if it is just
your home IP that has problems?

Cheers,
Andy

--
https://bitfolk.com/ -- No-nonsense VPS hosting

Romain

unread,

Sep 6, 2023, 5:10:05 AM9/6/23

to

So when this is happening mtr works but http, ssh and ping don't?

Yes

If you run tcpdump on the server, do your http/ssh packets even
reach it?

I wasn't seeing any request coming from my IP last time I checked. I'll re-check.

If you do tcptraceroute on port 22/80/443 from your home, how far
does it get?

Will test it next time it's blocked.

Are there any firewall rules in place on your server?

No (no fail2ban, ...).

DO you have access to any other host so you can tell if it is just
your home IP that has problems?

Yes, I have multiple servers (in OVH and outside OVH) that can all reach the server on IPv4. Same if I switch to my 5G connection, it works. Only my home's main IPv4 is blocked. IPv6 is OK.
I was even able to ask people using the same ISP to confirm, they can reach the IPv4 of my server.

Andy Smith

unread,

Sep 6, 2023, 9:30:07 AM9/6/23

to

On Wed, Sep 06, 2023 at 10:59:29AM +0200, Romain wrote:
> >
> > So when this is happening mtr works but http, ssh and ping don't?
>
> Yes

I think there is definitely a firewall involved somewhere as that is
quite complicated selective blocking: it's allowing back the ICMP
Time Exceeded packets that mtr uses for telling the path to a
destination, but it's not allowing ICMP Echo Reply packets that ping
uses for getting a response, nor of course any of the TCP packets
for http, ssh, etc.

If it was a routing issue I'd expect total loss of all connectivity,
but you don't see that, and as you later mention, you can still
reach the destination from other hosts and even from other
subscribers to your home ISP.

The tcptraceroute may help you find out where this firewall lives,
but it may end at or just before where you'd expect your host to be.

You are absolutely certain that no firewall rules exist on your
server?

# itables -nL

and/or

# nft list ruleset

Romain

unread,

Sep 6, 2023, 9:40:06 AM9/6/23

to

Next time it happens I'll run more tests from the server to my home.

I'm wondering if my modem at home could not be the culprit. I'm prepared for more tests here too.

But yes I'm sure I don't have any rules on the server:

# iptables -nL
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

# nft list ruleset
-bash: nft: command not found

Romain

unread,

Sep 7, 2023, 3:40:06 AM9/7/23

to

It happened again last night, but I wasn't able to investigate before I woke up, and for the moment it's not blocking anymore.
Out of curiosity, I'm doing a regular MTR, and I've had a strange thing happen.

A normal one (rpi4 at home to OVH):

└─# mtr -r 54.38.38.159 -4
Start: 2023-09-07T07:14:15+0000
HOST: rpi4 Loss% Snt Last Avg Best Wrst StDev
1.|-- livebox.home 0.0% 10 1.0 0.9 0.7 1.1 0.1
2.|-- 80.10.239.9 0.0% 10 3.0 2.9 2.7 3.5 0.3
3.|-- ae102-0.ncidf103.rbci.ora 0.0% 10 3.3 3.4 2.2 6.3 1.1
4.|-- ae51-0.nridf101.rbci.oran 0.0% 10 3.2 3.4 3.1 3.6 0.2
5.|-- ae41-0.noidf001.rbci.oran 0.0% 10 3.5 3.7 3.2 5.4 0.6
6.|-- be102.par-th2-pb1-nc5.fr. 0.0% 10 25.9 9.6 3.7 31.7 10.5
7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
9.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
10.|-- be103.rbx-g4-nc5.fr.eu 0.0% 10 8.1 9.0 7.2 20.9 4.2
11.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
12.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
13.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
14.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
15.|-- mail.borezo.info 0.0% 10 6.9 7.2 6.7 7.9 0.4

The same one a few minutes later (not normal):
└─# mtr -r 54.38.38.159 -4

Start: 2023-09-07T07:24:27+0000
HOST: rpi4 Loss% Snt Last Avg Best Wrst StDev
1.|-- livebox.home 0.0% 10 0.9 1.1 0.7 2.8 0.6
2.|-- 80.10.239.9 0.0% 10 2.8 3.0 2.7 4.4 0.5
3.|-- ae102-0.ncidf103.rbci.ora 0.0% 10 37.3 6.8 2.7 37.3 10.7
4.|-- ae51-0.nridf101.rbci.oran 0.0% 10 3.5 3.5 3.1 4.6 0.4
5.|-- ae41-0.noidf001.rbci.oran 0.0% 10 3.3 3.9 3.2 8.4 1.6
6.|-- be102.par-th2-pb1-nc5.fr. 0.0% 10 3.7 14.0 3.7 44.1 15.0
7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
9.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
10.|-- be103.rbx-g4-nc5.fr.eu 0.0% 10 7.5 8.4 7.1 12.1 1.7
11.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
12.|-- rpi4.home 90.0% 10 7955. 7955. 7955. 7955. 0.0

Paul van der Vlis

unread,

Sep 7, 2023, 4:00:07 AM9/7/23

to

Op 06-09-2023 om 15:40 schreef Romain:

> Next time it happens I'll run more tests from the server to my home.
> I'm wondering if my modem at home could not be the culprit. I'm prepared
> for more tests here too.

If you have switches at home, those could be the problem too.

You could let log iptables, in this way you can see if the packets reach
the machine.

With regards,
Paul

--
Paul van der Vlis Linux systeembeheer Groningen
https://vandervlis.nl

Romain

unread,

Sep 7, 2023, 4:40:06 AM9/7/23

to

I can reproduce with OVH IPs, but not with Scaleway IPs. I smell filtering on the OVH side.

Max Nikulin

unread,

Sep 7, 2023, 6:20:07 AM9/7/23

to

On 07/09/2023 14:31, Romain wrote:
> 12.|-- rpi4.home 90.0% 10 7955. 7955. 7955. 7955.
> 0.0

May it happen that you associated a local device with the IP of your
remote server? Check configuration of the .home DNS zone. Try to add -n
option to suppress DNS lookup and compare results.

Romain

unread,

Sep 7, 2023, 6:30:07 AM9/7/23

to

No, I'm using the DNS server included in my ISP modem, in which I can only select the last digits. It's impossible that an internal device has received a public IP address in the DNS zone (or it's not something I've done).

With -n (sometimes it stops at hop 7, sometimes 9):

└─# mtr -nr 54.38.38.159 -4
Start: 2023-09-07T08:17:12+0000

HOST: rpi4 Loss% Snt Last Avg Best Wrst StDev

1.|-- 192.168.0.1 0.0% 10 1.1 0.9 0.5 1.3 0.3
2.|-- 80.10.239.9 0.0% 10 3.2 3.3 2.3 5.3 0.9
3.|-- 193.253.80.138 0.0% 10 4.5 4.0 2.2 6.0 1.2
4.|-- 193.252.98.94 0.0% 10 3.4 4.3 3.1 12.2 2.8
5.|-- 193.252.98.101 0.0% 10 3.5 3.4 2.9 3.6 0.2
6.|-- 91.121.131.193 0.0% 10 4.0 12.4 3.7 82.6 24.7

7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0

9.|-- 192.168.0.2 90.0% 10 3461. 3461. 3461. 3461. 0.0

Andy Smith

unread,

Sep 7, 2023, 6:40:06 AM9/7/23

to

Hello,

On Thu, Sep 07, 2023 at 12:20:18PM +0200, Romain wrote:
> With -n (sometimes it stops at hop 7, sometimes 9):
> └─# mtr -nr 54.38.38.159 -4
> Start: 2023-09-07T08:17:12+0000
> HOST: rpi4 Loss% Snt Last Avg Best Wrst StDev
> 1.|-- 192.168.0.1 0.0% 10 1.1 0.9 0.5 1.3 0.3
> 2.|-- 80.10.239.9 0.0% 10 3.2 3.3 2.3 5.3 0.9
> 3.|-- 193.253.80.138 0.0% 10 4.5 4.0 2.2 6.0 1.2
> 4.|-- 193.252.98.94 0.0% 10 3.4 4.3 3.1 12.2 2.8
> 5.|-- 193.252.98.101 0.0% 10 3.5 3.4 2.9 3.6 0.2
> 6.|-- 91.121.131.193 0.0% 10 4.0 12.4 3.7 82.6 24.7
> 7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
> 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
> 9.|-- 192.168.0.2 90.0% 10 3461. 3461. 3461. 3461. 0.0

To me this suggests that the ICMP Time Exceeded packet is arriving
with source address 192.168.0.2, which I think means it is being
sent to you by your own ISP.

The way mtr works is to send an ICMP Echo Request packet to
54.38.38.159 with TTl set to 1 and see where the ICMP Time Exceeded
reply comes from, in this case 192.168.0.1. So it knows 192.168.0.1
is first hope. Then it does it again with TTL=2 and gets a reply
from 80.10.239.9. And so on.

So here when it does TTL=9 it gets a reply back from 192.168.0.2.

While it is possible that a string of providers are somehow routing
a packet that says it is from 192.168.0.2 back to you, really it is
most likely that this packet came from your own network or the thing
it is immediately connected to.

You may want to confirm with tcpdump that you receive a packet in on
your internet interface with source address 192.168.0.2.

In summary, I don't think it's OVH. I think it is your ISP or your
home router.

Max Nikulin

unread,

Sep 7, 2023, 7:00:06 AM9/7/23

to

On 07/09/2023 17:32, Andy Smith wrote:
> On Thu, Sep 07, 2023 at 12:20:18PM +0200, Romain wrote:
>> With -n (sometimes it stops at hop 7, sometimes 9):
>> └─# mtr -nr 54.38.38.159 -4
>> Start: 2023-09-07T08:17:12+0000
>> HOST: rpi4 Loss% Snt Last Avg Best Wrst StDev

^^^^

>> 9.|-- 192.168.0.2 90.0% 10 3461. 3461. 3461. 3461. 0.0
> To me this suggests that the ICMP Time Exceeded packet is arriving
> with source address 192.168.0.2, which I think means it is being
> sent to you by your own ISP.

I guess, these packets are generated by localhost = rpi4.home =
192.168.0.2. At first I did not noticed excessively large time in the
last line.

Concerning question related to .home, perhaps it is just /etc/hosts with
192.168.0.1 livebox.home
192.168.0.2 rpi4.home

I do not suspect some peculiarities in local configuration anymore.
tcpdump on the server may shed some light if packets from localhost
arrive to it.

zithro

unread,

Sep 7, 2023, 8:20:06 AM9/7/23

to

I'll write my answer here as well, as the OP posted the same posts on
debian-french (also top-posting ...).

Some ISPs or service providers may use private IPs (RFC1918) or even
APIPA for their internal routers, to spare public IPs.
CG-NAT (which uses APIPAs) especially may create some weird problems.

I think it's just a coincidence that the provider uses 192.168.0.2
internally and the OP host has the same address in its network.

--
++
zithro / Cyril

Romain

unread,

Sep 7, 2023, 5:50:07 PM9/7/23

to

I'm able to reproduce.

I can confirm that when this happens, it's the OVH server that fails to send the response to my network.

35 9.862648672 MY_PUBLIC_IP_AT_HOME → 54.38.38.159 ICMP 78 Echo (ping) request id=0x4b30, seq=33150/32385, ttl=1
36 9.862704895 54.38.38.159 → MY_PUBLIC_IP_AT_HOME ICMP 106 Destination unreachable (Port unreachable)

MTR from the OVH server to home:

Start: 2023-09-07T23:30:08+0200
HOST: rbx Loss% Snt Last Avg Best Wrst StDev
1.|-- 54.38.38.252 0.0% 10 0.6 0.5 0.3 0.7 0.1
2.|-- 10.162.250.98 0.0% 10 0.9 0.5 0.4 0.9 0.1
3.|-- 10.72.52.32 0.0% 10 0.5 0.5 0.4 0.7 0.1
4.|-- 10.73.17.42 0.0% 10 0.2 0.2 0.1 0.3 0.0
5.|-- 10.95.64.152 0.0% 10 1.1 1.2 1.1 1.5 0.1
6.|-- 54.36.50.226 0.0% 10 4.4 4.4 4.2 4.7 0.2
7.|-- 10.200.2.73 0.0% 10 78.0 11.6 4.1 78.0 23.4

8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0

Max Nikulin

unread,

Sep 8, 2023, 11:00:05 PM9/8/23

to

On 08/09/2023 04:39, Romain wrote:
> I can confirm that when this happens, it's the OVH server that fails to
> send the response to my network.
>
> 35 9.862648672 MY_PUBLIC_IP_AT_HOME → 54.38.38.159 ICMP 78 Echo (ping)
> request id=0x4b30, seq=33150/32385, ttl=1
> 36 9.862704895 54.38.38.159 → MY_PUBLIC_IP_AT_HOME ICMP 106 Destination
> unreachable (Port unreachable)

Some intermediate firewall may have specific rules for ICMP.

It is better to try tcpdump (its capture file may be later loaded to
wireshark) for HTTP and SSH packets at 54.38.38.159 and 192.168.0.2
rpi4.home, or if your router allows it, even better at 192.168.0.1
livebox.home.

Timothy M Butterworth

unread,

Sep 9, 2023, 12:50:07 AM9/9/23

to

On Fri, Sep 8, 2023 at 10:52 PM Max Nikulin <mani...@gmail.com> wrote:

On 08/09/2023 04:39, Romain wrote:
> I can confirm that when this happens, it's the OVH server that fails to
> send the response to my network.
>
> 35 9.862648672 MY_PUBLIC_IP_AT_HOME → 54.38.38.159 ICMP 78 Echo (ping)
> request id=0x4b30, seq=33150/32385, ttl=1
> 36 9.862704895 54.38.38.159 → MY_PUBLIC_IP_AT_HOME ICMP 106 Destination
> unreachable (Port unreachable)

Your hosting company may have PING flood protection. If you are running a constant PING from your PC to your server it could be detected as a PING flood. Are you able to SSH to the server while the PING is failing?

Some intermediate firewall may have specific rules for ICMP.

It is better to try tcpdump (its capture file may be later loaded to
wireshark) for HTTP and SSH packets at 54.38.38.159 and 192.168.0.2
rpi4.home, or if your router allows it, even better at 192.168.0.1
livebox.home.

--

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀

Romain

unread,

Sep 9, 2023, 4:40:06 AM9/9/23

to

As already reported, when it's blocked, all traffic is blocked on IPv4, including SSH & HTTP.