The default E-value for Tomtom is very relaxed as it is often used as a screening tool to spot general similarities to known motifs. The input motif is sometime "cleaned up" after spotting a partial match to a known motif.
When you get to the stage of wanting to claim a match to a known motif you should definitely use a more stringent E-value. An E-value of 0.05 or lower is the traditional threshold for statistical significance.