It sounds like you may have setup your Handle Server with a passphrase or with encryption turned on?
As noted in the docs, we recommend not using a passphrase. However, if you do need a passphrase, then you'll need to start your Handle server using the "net.handle.server.Main" script (provided by Handle.net) as described in the documentation at
https://wiki.lyrasis.org/display/DSDOC7x/Handle.Net+Registry+Support
If this doesn't help, then we may need you to send more information about your setup to this list (or to dspace-tech).
Tim