Neither of these file formats are representative of the actual ROM footprint. ELF contains plenty of metadata about the program, whereas HEX is encoded in ASCII, thus it has a significant overhead. Please export your firmware into a flat binary and then check its size.
Besides, if you're referring to the program you published in your previous post, you seem to be using exceptions, heap allocation, and heavy STL entities such as std::string. Removing that will reduce the size of the program drastically.
Hope this helps!
Pavel.