The documentation for (system) states:
"The resulting process writes to (current-output-port), reads from (current-input-port), and logs errors to (current-error-port)."
So, I'm guessing that it may be waiting for input. The following does not require an extra enter key:
(parameterize ([current-input-port (open-input-string "")])
(with-output-to-string (lambda () (system "ls"))))
Also, DrRacket doesn't require an extra enter key, so maybe there is something going on unique to the command line REPL.