Can iptables block TCP RST packets only?
James Taylor
I have discovered what I believe to be spoofed TCP reset packets being
injected into the stream to kill some large downloads I am trying to
make. I know the RST packets didn't come from the server I am
downloading from because I can see normal data-bearing packets arriving
after the RST packets and with later sequence numbers. It is probably
some half-wit attempt by the ISP to limit my downloads.
The problem is that when my computer receives the RST packets the
connection breaks and the download stops. If I could get my computer to
ignore these RSTs then I believe the connection would continue and my
downloads would complete.
Is there any way that I can use iptables to filter these RST packets
from a specific set of IP addresses corresponding to the servers in
question? A quick dip in the iptables man page has rather overwhelmed me
with the learning curve required, but also encouraged me when I saw
there was a --tcp-flags option.
Can anyone help me by showing me how to form a complete iptables command
line to block the RST packets?
--
James Taylor
Tony
wrote:
>Can anyone help me by showing me how to form a complete iptables command
>line to block the RST packets?
http://tuxtraining.com/2008/06/21/beating-sandvine-on-linux-with-iptables
If whoever is doing it, is doing it 'right' though, then you're in trouble
because they'll be sending RST's to both ends of the connection, as far as
I understand it.
Slashdot discussion,
http://tech.slashdot.org/article.pl?sid=08/06/30/0249249
--
Tony Evans
Saving trees and wasting electrons since 1993
blog -> http://perceptionistruth.com/
books -> http://www.bookthing.co.uk/
[ anything below this line wasn't written by me ]
Graham Murray
> I have discovered what I believe to be spoofed TCP reset packets being
> injected into the stream to kill some large downloads I am trying to
> make. I know the RST packets didn't come from the server I am
> downloading from because I can see normal data-bearing packets
> arriving after the RST packets and with later sequence numbers. It is
> probably some half-wit attempt by the ISP to limit my downloads.
Is it possible that by spoofing the IP address of the server you are
connected to that whoever is doing this is contravening the Computer
Misuse Act or some other legislation?
James Taylor
> James Taylor wrote:
>
>> Can anyone help me by showing me how to form a complete iptables command
>> line to block the RST packets?
>
> http://tuxtraining.com/2008/06/21/beating-sandvine-on-linux-with-iptables
Thanks. That's a starting point for further research at least.
There's still quite a lot to understand about iptables before I can see
how much of that information is relevant to me. I was rather expecting a
single command line to do the trick. Something semantically equivalent
to "drop all RST packets from IP address range xxx.xxx.xxx.xxx/24".
I suspect that eventually, I'll have to knuckle down and read the entire
documentation of iptables to master it myself, but it would be such a
huge help to me right now if an expert could give me a head start with a
specific command line.
> If whoever is doing it, is doing it 'right' though, then you're in trouble
> because they'll be sending RST's to both ends of the connection, as far as
> I understand it.
Yes, you'd expect them to send RSTs to both ends, but they seem to be
sending them only to my end because I can see plenty of normal data
packets arriving from the other end with later TCP sequence numbers than
are in the spoofed RSTs.
What I find strange is why they are reseting the connections rather than
just bandwidth throttling. Resets are destructive and force the user to
restart the download repeatedly until they manage to download it all. I
can continue incomplete downloads from where they left off, but this
often causes data corruption. I cannot check whether the download is
complete *and* uncorrupted until it has finished after many stops and
starts, and when I find that it is indeed corrupted I then have to
restart the whole download all over again, typically 3 or 4 times. All
this actually *adds* to the ISPs bandwidth burden whereas, if they
simply limited the download rate, they would get the intended result of
bandwidth reduction without severely inconveniencing their users.
--
James Taylor
James Taylor
I wish I knew enough about the law on this kind of thing because it is
appealing to think that I could throw the book at them. However, I
suspect that the ISP are within their rights to manage and balance data
flow across their network in line with the goal of providing reasonable
service to all users. To that end I could accept a degree of bandwidth
throttling. What I object to is having connections terminated entirely.
--
James Taylor
Tony
wrote:
>Tony wrote:
>
>> James Taylor wrote:
>>
>>> Can anyone help me by showing me how to form a complete iptables command
>>> line to block the RST packets?
>>
>> http://tuxtraining.com/2008/06/21/beating-sandvine-on-linux-with-iptables
>
>Thanks. That's a starting point for further research at least.
>
>There's still quite a lot to understand about iptables before I can see
>how much of that information is relevant to me. I was rather expecting a
>single command line to do the trick. Something semantically equivalent
>to "drop all RST packets from IP address range xxx.xxx.xxx.xxx/24".
<iptables command> -p tcp --dport 36745 --tcp-flags RST RST -j DROP;
What I'm not 100% sure about is whether it's an iptables -A INPUT, or
iptables -I FORWARD, I've seen various options and without going and
checking the manual myself, I'm not sure which to use. But that's the only
command you need (where 36745 is the port you're doing the transfer over)
James Taylor
> <iptables command> -p tcp --dport 36745 --tcp-flags RST RST -j DROP;
>
> What I'm not 100% sure about is whether it's an iptables -A INPUT, or
> iptables -I FORWARD, I've seen various options and without going and
> checking the manual myself, I'm not sure which to use. But that's the only
> command you need (where 36745 is the port you're doing the transfer over)
Ok, thanks. I'll check those options and get my head around it.
--
James Taylor
Mark Hobley
> I suspect that the ISP are within their rights to manage and balance data
> flow across their network in line with the goal of providing reasonable
> service to all users.
Throttling across their own network yes (or they could send a throttle control
request with their own IP address in it), but send spoofed packets (with a
falsified sender address) could be considered to be computer misuse.
Mark.
--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/
Mark Hobley
> What I'm not 100% sure about is whether it's an iptables -A INPUT, or
> iptables -I FORWARD, I've seen various options and without going and
> checking the manual myself, I'm not sure which to use.
It's an INPUT.
The FORWARD chain is mainly used for router and gateway applications.
alexd
> It is probably some half-wit attempt by the ISP to limit my downloads.
Have you considered changing ISP? If they're pulling stunts on the sly
like this, what other crazy shit are they up to [now or in the future]?
--
<http://ale.cx/> (AIM:troffasky) (UnSoEs...@ale.cx)
18:31:01 up 5 days, 23:22, 5 users, load average: 0.03, 0.40, 0.57
DIMENSION-CONTROLLING FORT DOH HAS NOW BEEN DEMOLISHED,
AND TIME STARTED FLOWING REVERSELY
Baron
For what its worth Talktalk / Opal Telecom do a similar thing with file
sharing sites ie "Rapidshare" etc.
--
Best Regards:
Baron.
Tony
> Tony<to...@darkstorm.invalid> wrote:
>> What I'm not 100% sure about is whether it's an iptables -A INPUT, or
>> iptables -I FORWARD, I've seen various options and without going and
>> checking the manual myself, I'm not sure which to use.
>
> It's an INPUT.
>
> The FORWARD chain is mainly used for router and gateway applications.
Thanks Mark / Dave, that makes eminent sense.
--
Tony Evans
Saving trees and wasting electrons since 1993
blog -> http://perceptionistruth.com/
books -> http://www.bookthing.co.uk
James Taylor
> James Taylor wrote:
>
>> Tony wrote:
>>
>>> http://tuxtraining.com/2008/06/21/beating-sandvine-on-linux-with-iptables
>>
>> There's still quite a lot to understand about iptables before I can see
>> how much of that information is relevant to me. I was rather expecting a
>> single command line to do the trick. Something semantically equivalent
>> to "drop all RST packets from IP address range xxx.xxx.xxx.xxx/24".
>
> <iptables command> -p tcp --dport 36745 --tcp-flags RST RST -j DROP;
>
> What I'm not 100% sure about is whether it's an iptables -A INPUT,
This was helpful as it guided my reading of the relevant sections of the
iptables man page. If I've understood it all correctly, the correct full
command line for blocking RST packets coming from the download servers
in my case is:
# iptables -A INPUT -p tcp -s 92.123.153.0/24 --sport 1935 \
--tcp-flags RST RST -j DROP
And I can check that the rule exists with:
# iptables -n -v -L INPUT
Chain INPUT (policy ACCEPT 436K packets, 526M bytes)
pkts bytes target prot opt in out source destination
114 4560 DROP tcp -- * * 92.123.153.0/24 0.0.0.0/0
tcp spt:1935 flags:0x04/0x04
The above report shows 114 matching packets during the course of a
download I tried, but unfortunately the resets are still getting through
the firewall... well, I can still see them in Wireshark and the
connections are still getting broken and having to be recontinued.
Can anyone tell me whether Wireshark *should* see the pre-firewall
traffic or not? How can I use Wireshark to see only the packets that
pass through the firewall?
--
James Taylor
Nigel Wade
Since you mention a firewall, is this rule really on a firewall?
If so, and the rule is meant to protect a machine behind the firewall,
then you need to add the rule to the FORWARD chain, not the INPUT chain.
The INPUT chain on the firewall is used to filter packets which terminate
at the firewall itself. The FORWARD chain is used for packets which come
in on one interface and exit on another.
The other point to note is that iptables rules do not work in isolation,
they are part of a chain. Packets are dealt with by the first rule they
match. You cannot just add/insert a rule into a chain without considering
the other rules which precede it. Another rule prior to the rule you've
added may be accepting the packets.
>
> Can anyone tell me whether Wireshark *should* see the pre-firewall
> traffic or not? How can I use Wireshark to see only the packets that
> pass through the firewall?
Yes, wireshark sees packets before they get to iptables module. If it's a
firewall, capture packets on the internal interface.
--
Nigel Wade
James Taylor
> James Taylor wrote:
>
>> If I've understood it correctly, the full command line for blocking
>> RST packets coming from the download servers in my case is:
>>
>> # iptables -A INPUT -p tcp -s 92.123.153.0/24 --sport 1935 \
>> --tcp-flags RST RST -j DROP
>>
>> And I can check that the rule exists with:
>>
>> # iptables -n -v -L INPUT
>> Chain INPUT (policy ACCEPT 436K packets, 526M bytes)
>> pkts bytes target prot opt in out source destination
>> 114 4560 DROP tcp -- * * 92.123.153.0/24 0.0.0.0/0
>> tcp spt:1935 flags:0x04/0x04
>>
>> The above report shows 114 matching packets during the course of a
>> download I tried, but unfortunately the resets are still getting through
>> the firewall... well, I can still see them in Wireshark and the
>> connections are still getting broken and having to be recontinued.
>
> Since you mention a firewall, is this rule really on a firewall?
There is no separate hardware firewall. I am refering to "iptables", the
software firewall on my Linux computer. I am trying to use iptables to
drop some spoofed TCP RST packets that are prematurely breaking some
large video downloads. I believe they are being spoofed by the ISP to
prevent me from downloading so much. I am within my rights to download
this legitimate media, so I want to bypass the ISPs meddling by
filtering out the spoofed RST packets. Am I wrong to use iptables for this?
> The INPUT chain on the firewall is used to filter packets which terminate
> at the firewall itself.
Yes, which is why the INPUT chain is the correct one to use in this case
I think. Please correct me if that's not right.
> The other point to note is that iptables rules do not work in isolation,
> they are part of a chain. Packets are dealt with by the first rule they
> match. You cannot just add/insert a rule into a chain without considering
> the other rules which precede it. Another rule prior to the rule you've
> added may be accepting the packets.
Well, if you look at the iptables -n -v -L INPUT command I show above
you'll see exactly what the INPUT chain contains; just one rule to drop
RST packets from that particular IP range. Did I do this incorrectly?
>> Can anyone tell me whether Wireshark *should* see the pre-firewall
>> traffic or not? How can I use Wireshark to see only the packets that
>> pass through the firewall?
>
> Yes, wireshark sees packets before they get to iptables module. If it's a
> firewall, capture packets on the internal interface.
My computer has only one Ethernet interface. How can I point Wireshark
at the already filtered traffic to check that iptables is working as
expected?
--
James Taylor
James Taylor
> Tony wrote:
>
>> If whoever is doing it, is doing it 'right' though, then you're in
>> trouble because they'll be sending RST's to both ends of the
>> connection, as far as I understand it.
>
> Yes, you'd expect them to send RSTs to both ends, but they seem to be
> sending them only to my end because I can see plenty of normal data
> packets arriving from the other end with later TCP sequence numbers than
> are in the spoofed RSTs.
Just to follow up, I found the above to be wrong. More careful study of
the TCP sequence numbers and packet arrival times reveals that the ISP
must be sending a RST packet to *both* ends.
I do see RST packets with sequence numbers earlier *in sequence* than
normal data packets, but these RSTs arrive later *in time* than all
normal data packets. The pattern seems to be like this: The first RST
that I see arrives immediately after the last data packet and it has a
sequence number placing it correctly after the data packet. Then, after
a short delay I see several RSTs with earlier sequence numbers
corresponding to the data packets that I have recently ACKed.
My conclusion is that the first RST is the spoofed one sent by the ISP
to tear down the connection, and the remaining RSTs are bounced by the
server itself when it gets my ACKs for data packets it has already sent
on a connection it now believes to be reset (because it was also sent a
spoofed RST packet by the ISP). Hope that makes sense.
--
James Taylor
Tony
wrote:
>James Taylor wrote:
>
>> Tony wrote:
>>
>>> If whoever is doing it, is doing it 'right' though, then you're in
>>> trouble because they'll be sending RST's to both ends of the
>>> connection, as far as I understand it.
>>
>> Yes, you'd expect them to send RSTs to both ends, but they seem to be
>> sending them only to my end because I can see plenty of normal data
>> packets arriving from the other end with later TCP sequence numbers than
>> are in the spoofed RSTs.
>
>Just to follow up, I found the above to be wrong. More careful study of
>the TCP sequence numbers and packet arrival times reveals that the ISP
>must be sending a RST packet to *both* ends.
Shame, can you tunnel the download over ssh or something?
James Taylor
> James Taylor wrote:
>
>> Just to follow up, I found the above to be wrong. More careful study of
>> the TCP sequence numbers and packet arrival times reveals that the ISP
>> must be sending a RST packet to *both* ends.
>
> Shame, can you tunnel the download over ssh or something?
Actually, due to the multi-faceted nature of the download setup process
I think it would have to be a VPN rather than a simple port-based ssh
tunnel.
The real problem is not the tunnelling, it's the fact that the tunnel
has to end somewhere so that the naked connection can continue on to the
server itself. I would be using both the upload and download capacity of
the VPN endpoint's Internet line fairly heavily so I'd need to own that
line or have a very friendly relationship with its owner. Which ISP
should I choose for this line and, given that my own ISP is monkeying
around with my downloads, what guarantee is there that the new ISP won't
do the same?
Maybe someone cleverer than me can see a technical solution, but I can't
see how you can bypass a middleman who sends RSTs to both ends, and I
can see no easy way to get a VPN endpoint in the UK with guaranteed
unfettered high-bandwidth low-cost Internet access.
--
James Taylor
Tom Anderson
> Tony wrote:
>
>> James Taylor wrote:
>>
>>> Just to follow up, I found the above to be wrong. More careful study of
>>> the TCP sequence numbers and packet arrival times reveals that the ISP
>>> must be sending a RST packet to *both* ends.
>>
>> Shame, can you tunnel the download over ssh or something?
>
> Actually, due to the multi-faceted nature of the download setup process I
> think it would have to be a VPN rather than a simple port-based ssh tunnel.
>
> The real problem is not the tunnelling, it's the fact that the tunnel has to
> end somewhere so that the naked connection can continue on to the server
> itself.
Like the server - any chance it will talk IPsec to you? Raw or over UDP
would solve this problem.
If the download is over HTTP, you could split it into N chunks, and
download each in parallel using a separate connection with a suitable
Range header. You could then throttle the connections (or let them be
throttled naturally) so that no single one of them is fat enough to
attract the attention of the strangler.
I have a vague memory that you said it was bittorrent. I would have
thought that would be fairly tolerant of killed connections. Maybe it
would benefit from having more connections, each running slower? I know BT
suffers from going too far that way, but it beats being stabbed.
> Maybe someone cleverer than me can see a technical solution, but I can't
> see how you can bypass a middleman who sends RSTs to both ends,
Prevent the RSTs from happening, or use UDP, which doesn't care.
tom
--
Tristan Tzara offered to create a poem on the spot by pulling words at
random from a hat. A riot ensued and Andre Breton expelled Tzara from
the movement.
James Taylor
> James Taylor wrote:
>
>> Tony wrote:
>>
>>> James Taylor wrote:
>>>
>>>> Just to follow up, I found the above to be wrong. More careful study
>>>> of the TCP sequence numbers and packet arrival times reveals that
>>>> the ISP must be sending a RST packet to *both* ends.
>>>
>>> Shame, can you tunnel the download over ssh or something?
>>
>> Actually, due to the multi-faceted nature of the download setup
>> process I think it would have to be a VPN rather than a simple
>> port-based ssh tunnel.
>>
>> The real problem is not the tunnelling, it's the fact that the tunnel
>> has to end somewhere so that the naked connection can continue on to
>> the server itself.
>
> Like the server - any chance it will talk IPsec to you? Raw or over UDP
> would solve this problem.
I didn't know IPsec could be wrapped in UDP. I guess doing this would
allow IPsec to work through NATs. Is that correct?
Is IPsec over UDP a recent innovation or has it always been part of the
way IPsec works?
Unfortunately, in this case I do not control the servers and thus have
to put up with the various TCP connections provided. I may be able to
tunnel out past my current ISP (assuming they don't just kill my tunnel)
but the real question is where to end the tunnel.
> If the download is over HTTP, you could split it into N chunks, and
> download each in parallel using a separate connection with a suitable
> Range header. You could then throttle the connections (or let them be
> throttled naturally) so that no single one of them is fat enough to
> attract the attention of the strangler.
That's an interesting approach. The question is how? Is there any
software I can get to automate this? Special wget or curl options?
Is there any way iptables can be used to throttle individual TCP flows?
> I have a vague memory that you said it was bittorrent.
No, BitTorrent was mentioned by someone else. My downloads are a mixture
of HTTP and RTSP. If it were possible to use BitTorrent then I don't
think I'd have any problem with connections getting broken; throttled
perhaps, but not broken.
>> Maybe someone cleverer than me can see a technical solution, but I
>> can't see how you can bypass a middleman who sends RSTs to both ends,
>
> Prevent the RSTs from happening, or use UDP, which doesn't care.
Neither are possible without changing the server, and that does not
belong to me. I can only use such techniques to tunnel and then we're
back to the problem of where to put the other end.
--
James Taylor
David Bolt
James Taylor painted this mural:
> Tom Anderson wrote:
>> If the download is over HTTP, you could split it into N chunks, and
>> download each in parallel using a separate connection with a suitable
>> Range header. You could then throttle the connections (or let them be
>> throttled naturally) so that no single one of them is fat enough to
>> attract the attention of the strangler.
>
> That's an interesting approach. The question is how? Is there any
> software I can get to automate this? Special wget or curl options?
Try aria2. It's able to use multiple servers, with multiple connections
to each server, while downloading files. You can find the sources here:
Regards,
David Bolt
--
Team Acorn: www.distributed.net OGR-NG @ ~100Mnodes RC5-72 @ ~1Mkeys/s
openSUSE 11.0 32b | | | openSUSE 11.3M1 32b
openSUSE 11.0 64b | openSUSE 11.1 64b | openSUSE 11.2 64b |
TOS 4.02 | openSUSE 11.1 PPC | RISC OS 4.02 | RISC OS 3.11
Nigel Wade
> Nigel Wade wrote:
>
>> James Taylor wrote:
>>
>>> If I've understood it correctly, the full command line for blocking
>>> RST packets coming from the download servers in my case is:
>>>
>>> # iptables -A INPUT -p tcp -s 92.123.153.0/24 --sport 1935 \
>>> --tcp-flags RST RST -j DROP
>>>
>>> And I can check that the rule exists with:
>>>
>>> # iptables -n -v -L INPUT
>>> Chain INPUT (policy ACCEPT 436K packets, 526M bytes) pkts bytes target
>>> prot opt in out source destination
>>> 114 4560 DROP tcp -- * * 92.123.153.0/24 0.0.0.0/0
>>> tcp spt:1935 flags:0x04/0x04
>>>
>>> The above report shows 114 matching packets during the course of a
>>> download I tried, but unfortunately the resets are still getting
>>> through the firewall... well, I can still see them in Wireshark and
>>> the connections are still getting broken and having to be recontinued.
>>
>> Since you mention a firewall, is this rule really on a firewall?
>
> There is no separate hardware firewall. I am refering to "iptables", the
> software firewall on my Linux computer.
Ok, the problem is one of ambiguity in the term "firewall". In it's
original meaning a firewall is an external device which protects internal
networks and has at least 2 interfaces. But the term has been abused by
vendors of both software "firewalls" and mock "firewalls" in routers,
that the term now conveys little meaning, and much confusion, without
additional information.
> I am trying to use iptables to
> drop some spoofed TCP RST packets that are prematurely breaking some
> large video downloads. I believe they are being spoofed by the ISP to
> prevent me from downloading so much. I am within my rights to download
> this legitimate media, so I want to bypass the ISPs meddling by
> filtering out the spoofed RST packets. Am I wrong to use iptables for
> this?
Yes, I understand your intent from the thread. Blocking the RST packets
may do what you want, but RST packets are used for perfectly valid and
legitimate reasons on normal connections, so throwing them all away may
result in other, strange issues.
The question is, are you absolutely certain that these RST packets are
being inserted by your ISP? Have you tried contacting your ISP for an
explanation?
They could be perfectly legitimate RST packets sent by the other end if
that end has prematurely closed the socket (maybe the other end has
crashed). If your connection is going via a NAT router the router may
flush the NAT entry after a certain amount of inactivity, and it might
send RSTs to ensure the connection is dropped - or it might be a bug in
the routers NAT implementation. The other end could be using iptables and
flushing/loading the rules or in some other way causing problems to the
traffic flow e.g. flushing then loading new rules when
ESTABLISHED,RELATED is not set might cause existing connections to have
RST sent (I don't know this for certain, but it might be a cause).
>
>> The INPUT chain on the firewall is used to filter packets which
>> terminate at the firewall itself.
>
> Yes, which is why the INPUT chain is the correct one to use in this case
> I think. Please correct me if that's not right.
Yes, in your context you want the INPUT chain.
>
>> The other point to note is that iptables rules do not work in
>> isolation, they are part of a chain. Packets are dealt with by the
>> first rule they match. You cannot just add/insert a rule into a chain
>> without considering the other rules which precede it. Another rule
>> prior to the rule you've added may be accepting the packets.
>
> Well, if you look at the iptables -n -v -L INPUT command I show above
> you'll see exactly what the INPUT chain contains; just one rule to drop
> RST packets from that particular IP range. Did I do this incorrectly?
Looks ok to me, if that's the only rule.
>
>>> Can anyone tell me whether Wireshark *should* see the pre-firewall
>>> traffic or not? How can I use Wireshark to see only the packets that
>>> pass through the firewall?
>>
>> Yes, wireshark sees packets before they get to iptables module. If it's
>> a firewall, capture packets on the internal interface.
>
> My computer has only one Ethernet interface. How can I point Wireshark
> at the already filtered traffic to check that iptables is working as
> expected?
I don't know any way to do that.
You may want to log the dropped packets before you discard them. At least
that way you'd know that the rule is actually being invoked.
Is there any way to monitor the connection on the other end, and
determine what's really going on?
--
Nigel Wade
James Taylor
> Try aria2. It's able to use multiple servers, with multiple connections
> to each server, while downloading files. You can find the sources here:
>
> http://aria2.sourceforge.net/
Ok, thanks, I'll look into that.
--
James Taylor
James Taylor
> James Taylor wrote:
>
>> There is no separate hardware firewall. I am refering to "iptables", the
>> software firewall on my Linux computer.
>
> Ok, the problem is one of ambiguity in the term "firewall". In it's
> original meaning a firewall is an external device which protects internal
> networks and has at least 2 interfaces. But the term has been abused by
> vendors of both software "firewalls" and mock "firewalls" in routers,
> that the term now conveys little meaning, and much confusion, without
> additional information.
Precise use of terms is always preferable but rarely can you defeat the
tide of common usage. If "firewall" is an inappropriate term, what would
you describe iptables as? You could probably call it a "packet filter"
with reasonable accuracy, but that term seems even more vague.
> Blocking the RST packets may do what you want, but RST packets are
> used for perfectly valid and legitimate reasons on normal connections,
> so throwing them all away may result in other, strange issues.
Well it results in some connections timing out instead of terminating
immediately. If there were a lot of simultaneous connections this might
become a resource hog, but in my case there are not so many and I can
live with this timeout behaviour during downloads. Also, note that I am
only blocking RST packets from the IP range of the servers in question
which happens to be a /24 block owned by Akamai.
> The question is, are you absolutely certain that these RST packets are
> being inserted by your ISP?
No I cannot be certain that it is them. It could be some other
middleman. It seems unlikely that Akamai's CDN severs themselves would
be terminating connections prematurely. I don't see what their
motivation would be and they have more than enough capacity. I'm not
sure who else would care. Only in the case of my ISP can I see a motive.
> Have you tried contacting your ISP for an explanation?
Haha! Have you ever tried asking an ISP technical questions? They never
seem to have people who know what they're doing available to customers.
Instead they have support monkeys who assume that if they don't
understand your question then it must be *you* that's clueless, and they
make little attempt to disguise their contempt. Drives me nuts!
> They could be perfectly legitimate RST packets sent by the other end if
> that end has prematurely closed the socket (maybe the other end has
> crashed).
It is certainly possible that Akamai might terminate all connections
with a RST instead of the normal FIN handshake, just because it's a few
less packets. But this isn't the full explanation because the files
being downloaded are not complete when the connections are reset and so
downloads must be resumed multiple times until they are complete.
> If your connection is going via a NAT router the router may
> flush the NAT entry after a certain amount of inactivity,
But there is no inactivity during the course of a download, of course.
> and it might send RSTs to ensure the connection is dropped - or it
> might be a bug in the routers NAT implementation.
I've had the same router for years and not encountered this kind of
thing before. Also, the downloads were all working fine a week or so
ago. These connection resets only started happening a couple of days
before I started this thread.
> The other end could be using iptables and flushing/loading the rules
> or in some other way causing problems to the traffic flow e.g.
> flushing then loading new rules when ESTABLISHED,RELATED is not set
> might cause existing connections to have RST sent (I don't know this
> for certain, but it might be a cause).
Well, you certainly seem to know more about iptables than I do.
What exactly is "ESTABLISHED,RELATED", and what does it mean to be
flushing and loading rules?
> You may want to log the dropped packets before you discard them. At least
> that way you'd know that the rule is actually being invoked.
Sounds like a good idea. I'm new to iptables, would you be kind enough
to show me the best way to do that kind of logging?
> Is there any way to monitor the connection on the other end, and
> determine what's really going on?
I can't see how I would do that short of getting a job with Akamai.
--
James Taylor
Darren Salt
[snip]
> Precise use of terms is always preferable but rarely can you defeat the
> tide of common usage. If "firewall" is an inappropriate term, what would
> you describe iptables as? You could probably call it a "packet filter" with
> reasonable accuracy, but that term seems even more vague.
It isn't that either.
It's a convenient utility for viewing and altering netfilter's IPv4
configuration.
[snip]
--
| Darren Salt | linux at youmustbejoking | nr. Ashington, | Doon
| using Debian GNU/Linux | or ds ,demon,co,uk | Northumberland | Army
| + http://www.youmustbejoking.demon.co.uk/ & http://tartarus.org/ds/
Sorry, you'll have to type up. I can't read you.
Tony Mountifield
James Taylor <use...@oakseed.demon.co.uk.invalid> wrote:
> There is no separate hardware firewall. I am refering to "iptables", the
> software firewall on my Linux computer. I am trying to use iptables to
> drop some spoofed TCP RST packets that are prematurely breaking some
> large video downloads. I believe they are being spoofed by the ISP to
> prevent me from downloading so much. I am within my rights to download
> this legitimate media, so I want to bypass the ISPs meddling by
> filtering out the spoofed RST packets. Am I wrong to use iptables for this?
There may be another complication, if you are using an external router
that provides NAT service (e.g. the router has a public IP on its WAN
interface, but all your internal boxes are on 192.168.x.x).
In that situation, the router has to maintain a session table of all
open connections, so that it can translate the private IP:port of a
connection into a corresponding public IP:port. If the router receives
a RST packet from the remote end, then as well as translating it and
passing it on to the internal host, it might also mark the session as
terminated, causing the router to stop passing data packets through.
I've never had reason to investigate or observe such behaviour, but it
seems quite possible to me that it might occur.
Cheers
Tony
--
Tony Mountifield
Work: to...@softins.co.uk - http://www.softins.co.uk
Play: to...@mountifield.org - http://tony.mountifield.org
Tom Anderson
> Tom Anderson wrote:
>> James Taylor wrote:
>>> Tony wrote:
>>>> James Taylor wrote:
>>>>
>>>>> Just to follow up, I found the above to be wrong. More careful study of
>>>>> the TCP sequence numbers and packet arrival times reveals that the ISP
>>>>> must be sending a RST packet to *both* ends.
>>>>
>>>> Shame, can you tunnel the download over ssh or something?
>>
>> Like the server - any chance it will talk IPsec to you? Raw or over UDP
>> would solve this problem.
>
> I didn't know IPsec could be wrapped in UDP. I guess doing this would
> allow IPsec to work through NATs. Is that correct?
Yes. And firewalls that block protocols that aren't TCP and UDP.
> Is IPsec over UDP a recent innovation or has it always been part of the
> way IPsec works?
Pass. The Cisco VPN client has done it for ages, and it's possible it's
largely a Ciscoism, although i believe it is a standard.
> Unfortunately, in this case I do not control the servers and thus have
> to put up with the various TCP connections provided. I may be able to
> tunnel out past my current ISP (assuming they don't just kill my tunnel)
> but the real question is where to end the tunnel.
Oh well, no point worrying about that, though.
>> If the download is over HTTP, you could split it into N chunks, and
>> download each in parallel using a separate connection with a suitable Range
>> header. You could then throttle the connections (or let them be throttled
>> naturally) so that no single one of them is fat enough to attract the
>> attention of the strangler.
>
> That's an interesting approach. The question is how? Is there any
> software I can get to automate this? Special wget or curl options?
There are almost certainly wget or curl options for this. Man page time,
i'm afraid.
> Is there any way iptables can be used to throttle individual TCP flows?
Pass. But there's definitely a throttle option on at least one of wget and
curl.
>>> Maybe someone cleverer than me can see a technical solution, but I can't
>>> see how you can bypass a middleman who sends RSTs to both ends,
>>
>> Prevent the RSTs from happening, or use UDP, which doesn't care.
>
> Neither are possible without changing the server, and that does not
> belong to me. I can only use such techniques to tunnel and then we're
> back to the problem of where to put the other end.
Throttling your connections will (i hope) prevent the RSTs from happening.
tom
--
Hesgadin. It was in two parts - both of them silent. I remember this map
came with a letter accusing me of stealing eggs. I had never understood
the relationship of the map to the accusation. I still don't, but I'm
grateful for the map.
James Taylor
> James Taylor wrote"
>
>> Precise use of terms is always preferable but rarely can you defeat the
>> tide of common usage. If "firewall" is an inappropriate term, what would
>> you describe iptables as? You could probably call it a "packet filter" with
>> reasonable accuracy, but that term seems even more vague.
>
> It isn't that either.
>
> It's a convenient utility for viewing and altering netfilter's IPv4
> configuration.
Aha, yes of course, but then the question becomes what to call netfilter?
PS. Hi Darren, its a small world, in Usenet land anyway. Are you still
using RISC OS, or are you entirely Linux now? I had to abandon RISC OS
(kicking and screaming) years ago when I left the UK for a life of
travelling with a laptop, and frankly I'm now rather glad to be a Linux
user, although I've never reclaimed on this platform the assembler and
GUI programming skill level I had on RISC OS.
--
James Taylor
James Taylor
> James Taylor wrote:
>
>> I am trying to use iptables to drop some spoofed TCP RST packets
>> that are prematurely breaking some large video downloads. I believe
>> they are being spoofed by the ISP to prevent me from downloading so
>> much. I am within my rights to download this legitimate media, so I
>> want to bypass the ISPs meddling by filtering out the spoofed RST
>> packets. Am I wrong to use iptables for this?
>
> There may be another complication, if you are using an external router
> that provides NAT service (e.g. the router has a public IP on its WAN
> interface, but all your internal boxes are on 192.168.x.x).
Yes, I am.
> In that situation, the router has to maintain a session table of all
> open connections, so that it can translate the private IP:port of a
> connection into a corresponding public IP:port. If the router receives
> a RST packet from the remote end, then as well as translating it and
> passing it on to the internal host, it might also mark the session as
> terminated, causing the router to stop passing data packets through.
Actually the NAT table entry does not timeout for some while. It
certainly doesn't get wiped the very moment it sees a RST.
--
James Taylor
James Taylor
> James Taylor wrote:
>
>> Is IPsec over UDP a recent innovation or has it always been part of
>> the way IPsec works?
>
> Pass. The Cisco VPN client has done it for ages, and it's possible it's
> largely a Ciscoism, although i believe it is a standard.
Ah, now that you mention it, I recall I've seen this from a Cisco VPN.
>> That's an interesting approach. The question is how? Is there any
>> software I can get to automate this? Special wget or curl options?
>
> There are almost certainly wget or curl options for this. Man page time,
> i'm afraid.
Hmmm, I don't remember seeing any for multi-server multi-connection
downloads but I will check. Another issue is that not all of the
downloads are over HTTP. Some are RTSP/RTMP and that may not be
supported by either tool.
>> Is there any way iptables can be used to throttle individual TCP flows?
>
> Pass. But there's definitely a throttle option on at least one of wget
> and curl.
Hmmm, now that you mention it, I do recall seeing a throttling option in
one of them (wget I think). That could be useful. However, it would
certainly be much easier to throttle the downloads with iptables (or
some other filter utility) than it would be to hack the source code of
the downloader utility I'm using to replace every type of connection
with a wget/curl equivalent. Indeed that may not even be possible.
> Throttling your connections will (i hope) prevent the RSTs from happening.
If I can find a simple way to throttle, I'm certainly keen to try it.
--
James Taylor
Darren Salt
> Darren Salt wrote:
>> James Taylor wrote"
>>> Precise use of terms is always preferable but rarely can you defeat the
>>> tide of common usage. If "firewall" is an inappropriate term, what would
>>> you describe iptables as? You could probably call it a "packet filter"
>>> with reasonable accuracy, but that term seems even more vague.
>> It isn't that either.
>> It's a convenient utility for viewing and altering netfilter's IPv4
>> configuration.
> Aha, yes of course, but then the question becomes what to call netfilter?
See http://www.netfilter.org/ ;-)
> PS. Hi Darren, its a small world, in Usenet land anyway. Are you still
> using RISC OS, or are you entirely Linux now?
The RISC OS hw's still here, but...
> I had to abandon RISC OS (kicking and screaming) years ago when I left the
> UK for a life of travelling with a laptop, and frankly I'm now rather glad
> to be a Linux user, although I've never reclaimed on this platform the
> assembler and GUI programming skill level I had on RISC OS.
x86 asm I avoid. :-)
--
| Darren Salt | linux at youmustbejoking | nr. Ashington, | Doon
| using Debian GNU/Linux | or ds ,demon,co,uk | Northumberland | Army
| + http://wiki.debian.org/DebianEeePC/
You will be surrounded by luxury.
James Taylor
> x86 asm I avoid. :-)
Too right!
--
James Taylor
alexd
> On Wed, 24 Feb 2010, James Taylor wrote:
>> Is IPsec over UDP a recent innovation or has it always been part of the
>> way IPsec works?
>
> Pass. The Cisco VPN client has done it for ages, and it's possible it's
> largely a Ciscoism, although i believe it is a standard.
I've seen port 4500/UDP used by other vendor's VPN clients. I believe
it's referred to as IPSEC NAT-T.
--
<http://ale.cx/> (AIM:troffasky) (UnSoEs...@ale.cx)
21:59:00 up 10 days, 2:50, 5 users, load average: 0.03, 0.03, 0.00
Tom Anderson
> Tom Anderson wrote:
>
>> James Taylor wrote:
>>
>>> That's an interesting approach. The question is how? Is there any software
>>> I can get to automate this? Special wget or curl options?
>>
>> There are almost certainly wget or curl options for this. Man page time,
>> i'm afraid.
>
> Hmmm, I don't remember seeing any for multi-server multi-connection downloads
> but I will check.
Ah, i was thinking more that if there are options for ranges and
throttling, you could moderately trivially build a multi-connection
downloader with shell script and arithmetic.
> Another issue is that not all of the downloads are over HTTP. Some are
> RTSP/RTMP and that may not be supported by either tool.
Yhea, not the faintest idea how you could do something comparable with
that!
>>> Is there any way iptables can be used to throttle individual TCP flows?
>>
>> Pass. But there's definitely a throttle option on at least one of wget and
>> curl.
>
> Hmmm, now that you mention it, I do recall seeing a throttling option in
> one of them (wget I think). That could be useful. However, it would
> certainly be much easier to throttle the downloads with iptables (or
> some other filter utility) than it would be to hack the source code of
> the downloader utility I'm using to replace every type of connection
> with a wget/curl equivalent. Indeed that may not even be possible.
Yes, if you can't easily extract a stream of URLs, a curl/wget approach
isn't applicable.
If a firewall won't do the throttling, you could try a proxy. Squid
certainly does throttling.
But apparently there's something called tc which is will do throttling in
what seems to be an astonishingly configurable way.
tom
--
Miscellaneous Terrorists: Ducks | Bird Flu | Avian flu | Jimbo Wales |
Backstreet Boys | The Al Queda Network | Tesco -- Uncyclopedia