One thing to note is that the media *also* have display modes. By default the display modes are all configured to display the item or load a media player (with the exception of the File media). You can create a new display mode for media called something like "Download" and then add whatever fields to that display mode for each media that you would like. Further, each file type has their own formatters, so the audio file has a generic link formatter, but there aren't any image formatters provided by default that simply display a link. So, @ UNLV, we use the file_download module (
https://www.drupal.org/project/file_download) and use that formatter for all of our download media display modes. Then you can either modify the existing Original File EVA to use the new media display mode you created, or create a whole new one to add to your Node's display modes.
I hope that was clear and helpful. We can also chat on the Islandora Slack if you want to talk through it.