Bear in mind that the voltage rating specified will be under the rated
load.
Unloaded, the output voltage will be higher than the full-load rating.
How much higher it will be can vary a lot from one wall wart to the
next.
Not sure what application would be that critical about 4 Volts. As for
using a resistor, this is only effective for a fixed load current. If
the actual current drain will change while it's in use, the resistor's
voltage drop will also change.
Speaking of which, you have not specified the power/current
requirement this transformer should provide.
Makes a difference.
73