The UNITS menu is "sticky", i.e. it stays active until explicitly dismissed by pressing EXIT.
The HP-42S has six sticky menus: the "application" menus SOLVER, ∫f(x), MATRIX, STAT, and BASE, and the CUSTOM menu. Free42 inherits the HP-42S menu system, and Plus42 adds two more sticky menus: DIRS and UNITS.
The application menus have the highest level of stickiness, in that they remain active even after other, non-application menus have been selected and dismissed. DIRS and units are at the medium level of stickiness, and CUSTOM and the function menus (when made sticky by activating them twice) are at the lowest level.
That's a lot more detail than you asked for, but I think it is worth mentioning. The short answer is basically that you can always get out of menus by pressing EXIT, but you may have to press EXIT multiple times in order to get out of nested menus and/or menus with different levels of stickiness layered on top of each other. But regardless of the intricacies of the menu system, as long as you remember to use EXIT to get out of menus, you'll be fine.