I see. Well, in that case, time would not be unique enough. You could do a union of timestamp+TCP sequence number. But, even that might not be unique enough. You could still have a collision with that formula. Ultimately, it depends on what you care about and how you want to be able to recall the information. What is the most relevant piece of information in the packet? Source port? Destination port? TCP Flag? You might want the key to be a union of the most relevant pieces of information.
Perhaps if you explained the use case a bit, we could recommend an ideal way of storing and recalling the information.
Brendan