Yes, you can web-host like a regular static webpage. The Emscripten compiler usually outputs three files: one .html, one .js and one .wasm. The .html file is basically your index.html which is loaded when visiting the URL. The .js and .wasm files are then loaded by the .html file, the only rule is that the .js and .wasm file must be in the same web server directory as the .html file.
I'm just host my WASM stuff on GH Pages, for instance:
The visual6502remix page hosting directory looks like this (just the output of emcc, with the .html file renamed to index.html):
Be aware though that hosting on Github Pages means that some WASM features like multi-threading (or rather SharedArrayBuffer) are disabled, because the GH Pages web server can't be reconfigured to return the required response headers (there's a workaround with service workers which I haven't tried yet). Other web hosters allow this sort of configuration, so the choice of web hoster dictates what sort of WASM features you can use (which IMHO is a massive design wart, but whatever...)
Cheers!