Steve's $readmemh approach should be pretty efficient. A non-blackbox approach that I was using was to add a DPI function inside of the Chisel-emitted Verilog, `dpi_readmemh` that takes a filename as input. From the C++ testbench I could then load then initialize that memory with whatever I want. At the time, I wrote a Perl script that would apply this instrumentation for a specific signal in a specific module. Note: this is horribly kludgy and the best approach here, I expect, would be to enable this with a FIRRTL pass that exposes memories via the DPI/VPI.
Note: for the VPI/DPI, I've found Verilator to be horrendously lacking in documentation...
At a more philosophical level, this is one of the weird disconnects between what Chisel is doing and what you think Chisel should be capable of. Verilog/System Verilog gives you 100% signal visibility in a completely non-synthesiable way. If you want to peek/poke some signal way deep inside of a module that has no IO to do so, you can. Chisel, on the other hand, is restricting you to a set synthesizable constructs as it's really a language for emitting circuit descriptions in FIRRTL. From the purely Chisel level, dynamically loading memory should be impossible unless you emit a circuit that allows you to do this.