I have been wondering if wearing a wetsuit under your drysuit is beneficial. The wetsuit works by trapping water in itself which is then heated by your bodies temperature. Because of it's cell structure water does not move quickly through it therfore keeping you warm unless you spend alot of time in the cold water. That is why you should dunk yourself in water to start the process, or take a warm shower with it on if you can.
I believe thermal type clothing (layering) will be much more beneficial as I believe someone already mentioned. It will also help movement of limbs and help the circulation of blood.
I haver heard of fisherman wearing waiters to stay dry also carried sharp knives to slice open the bottom of the waiters in case they fell in and could not get out of the water because of the weight.
And never go out by yourself in weather like this. One small mistake could lead to catastrophical situations really quick.
Can you swim to shore with in a few minutes if you had to? Hypothermia sets in pretty quick in these temps especialy with the wind blowing, and there is no one able to rescue you in time if something bad did happen, God forbid.