file tranfer

[Thomas Czekalla]

Is there a best-practice for transferring files (>1mb)?

Does this sound like a reasonable design strategy?

...
byte[] payload = new byte[1048576]; // 1mb
while ((read = ios.read(payload)) != -1) {
    FileMessage fm = new FileMessage();
    fm._payload = payload;
    fm._fileName = fileName;
    if (ios.available() > 0) {
        fm._isEof = false;
    } else {
        fm._isEof = true;
    }
    sc.publish(subject, serialize(fm), acb);
}
...

The alternative is to generate a protocol with google protobuf and then send over nats.

All the examples I have seen in the unit tests and documentation are just sending small text messages.

[Ivan Kozlovic]

You would need to take into account the overhead of your serialization into the max payload. So use less for the byte array itself.

Also, you may want to include a sequential ID in each chunk so that the receiving end can fail if it detects a gap (since messages can be lost).

Hope this helps.

Ivan.

[tyler...@apcera.com]

Is there a reason you can't store the file in a blob store, e.g. Amazon S3 or Google Cloud Storage, and then pass the file pointer via NATS? This is the pattern I generally recommend for file transfer.