How are multiple, concurrent TCP Streams handled?

[Richard Howell-Peak]

I have a quick question about how the TCP Stream class works.

At the moment, it's doing what I was expecting, which is reassembling tcp streams so I can gather up all the parts of say, a large png file that is part of an http request / response.

What I'm not sure however, is how the library will behave when there are multiple http requests and responses going over the network.  For example if someone tried to load several pages at once.

Would we get a situation like the following (imagine this is a flow of packets coming back through the stream callbacks and being fed into my code):

[ Request URL # 1]

[ Request URL # 2]

[ Request URL # 3]

[ Response to URL # 1 part 1]

[ Response to URL # 1 part 2]

[ Response to URL # 1 part 3]

[ Response to URL # 2 part 1]

[ Request URL # 4]

[ Response to URL # 2 part 2]

[ Response to URL # 3 part 1]

[ Response to URL # 3 part 2]

[ Response to URL # 4 part 1]

[ Response to URL # 4 part 2]

[ Response to URL # 4 part 3]

Which would be awesome.  Now I'm pretty sure this would happen if the multiple requests were all sent to the same server, but if not, is something like this likely to occur:

[ Request URL # 1 at server A]

[ Request URL # 2 at server B]

[ Request URL # 3 at server C]

[ Response to URL # 1 part 1]

[ Response to URL # 2 part 1]

[ Response to URL # 3 part 1]

[ Response to URL # 1 part 2]

[ Response to URL # 1 part 3]

[ Response to URL # 2 part 2]

[ Response to URL # 3 part 2]

[ Response to URL # 4 part 1]

[ Response to URL # 4 part 2]

Where server B starts sending back his data before server A has finished sending the rest of the packets for the first request there.  How would libtins handle that?  From what I read about the StreamFollower the idea is it will separate out the different TCP streams (and I would think that is based on ip and port number etc), so would each server get it's own TCP Stream somehow?  And if so how is that implemented in the code?

If not, I suppose I'll need to write my own code to check on the source and destination ip and port numbers and figure it all out myself!

[Matias Fontanini]

Each TCP stream is reassembled individually. Whatever happens on each stream, libtins will just reassemble it and then execute the callback once there's available data on them. If a browser is performing multiple requests over the same connection, then all requests will be serialized, so it would look like your first scenario. If your browser is connecting to multiple servers then it would look like your second scenario. Still, each stream will be reassembled individually, so you will get 3 streams, once for each server and all data you receive on each stream will be in order (meaning you'll get parts 1,2 and 3 for each stream). So yeah, I don't know exactly what you're trying to do, but every stream will be represented by a different Stream object.

You can check the way this works if you want. This is on tcp_ip/stream_follower.h. Each stream is identified by the 4 tuple (source_ip, source_port, destination_ip, destination_port) and each packet that belongs to a stream is processed individually in the context of each stream.

[Richard Howell-Peak]

Ok fine, so lib tins will basically create a new stream object and when I call client_data_callback it will forward all the packets for that stream into that particular function, so it'll be running several copies of the code concurrently.

Then when the stream stops it will destroy the stream object.

I think what I'd have to do then to make sure the packets don't get mixed up (essentially what I'm trying to do is reassemble the original png or jpg sent over http), is somehow create a handler object that gets created when a new stream is created so that each stream has it's own handler attached to it, rather than have all the streams using the same static callback (which is what I'm doing at the moment).

[Richard Howell-Peak]

I noticed in the docs there is a TCPStream object but the tutorial has Stream object.

Also the docs have TCPStream.id() function which looks handy (but won't compile saying it's not a member of Stream).  Is the ID function guaranteed to be unique?  

[Matias Fontanini]

There's 2 set of classes. TCPStreamFollower was the first, crappy implementation that I wrote for some other reason and then decided to add it to the library. The classes inside namespace Tins::TCPIP are the new ones which you should use. In there you'll find StreamFollower, Stream, etc. You really don't need to do anything to differentiate different streams. StreamFollower already reassembles the streams so you get the stream's data in order. That's the whole point of the class.

The id in streams is just the 4 tuple (source address, source port, destination address, destination port), sorted in such a way that packets in any direction will always map to the same id. This is not unique and it's not meant to be; it's just a way to map a packet to a new/existing stream.