The button is not rendered in a non-Runestone HTML build because the option makes no sense if there is no server infrastructure to handle the upload.
Unfortunately, yes, that means there is no way to preview the page outside of setting up a Runestone server to build it on.