Building ns-3 takes a fair amount of time if you run
./ns3 configure --enable-examples --enable-tests && ./ns3 build
This is because you are building everything. If you run
./ns3 configure --enable-examplese --enable-python-bindings && ./ns3 run first.py
It will build only the libraries and then run the python script.
Loading the bindings can be pretty slow (from ns import ns).
Not much I can do to speed it up.
It runs an awful lot of stuff from cling and cppyy to build the bindings at runtime and make it available.
I haven't tried caching it to disk and then loading, but may be an option (like shader caches used in games, where you compile once and load later).
Regarding the NS3_BINDINGS_INSTALL_DIR. Don't touch that. It is meant for repackagers, not end-users.