Nathan, first make sure you're using the Tooling API for deployment as much as possible. Look under
Settings>Illuminated Cloud>Validation and Deployment and make sure that all of the
Prefer Tooling API for options are checked. That will directly mimic the deployment path of Developer Console and shouldn't generally deploy any slower (ignoring network transfer times for larger files).
If it's still deploying slowly, please add the following under Help>Debug Log Settings:
#com.illuminatedcloud.intellij.builder.ForceComBuilder
#com.illuminatedcloud.intellij.builder.ForceComToolingApiDeployer
#com.illuminatedcloud.intellij.builder.ForceComMetadataApiDeployer
perform a slow deployment, and send me the resulting logs (Help>Show Log in Explorer/Finder). That should show me the timings of the various API calls for deployment operations.
Assuming it's what I've seen in the recent past...basically just waiting for the server to say that the asynchronous operation is complete...you'll likely want to open a support case with Salesforce with the details. Let's confirm that one way or the other first, though.
Regards,
Scott