How to get InputStream and OutputStream from Cloud Storage on App Engine Java?

[Richard Arnold]

I am attempting to use FFMPEG on Google App Engine (Java) to re-encode some video files I have that are sitting in Google Cloud Storage. I found a Java wrapper around ffmpeg that I would like to use here: https://github.com/kokorin/Jaffree. 

My issue is that this library requires an InputStream and OutputStream. I am trying to specify the InputStream as a file in Google Cloud Storage and the OutputStream as a new file. 

I think I have done specifying the InputStream correctly, but I do not know what to do about the OutputStream.

How can I specify the OutputStream to be a new file?

Or if possible, can I just overwrite the file in Google Cloud Storage?

Here is my Java code in Google App Engine so far:

Storage storage = StorageOptions.getDefaultInstance().getService();

// This is the input file

BlobId id = BlobId.of("storageBucket", "videos/video_1.mp4");

byte[] content = storage.readAllBytes(id);

InputStream inputStream = new ByteArrayInputStream(content);

FFmpeg.atPath()

    .addInput(PipeInput.pumpFrom(inputStream))

    .setOverwriteOutput(true)

    .addArguments("-movflags", "faststart")

    .addOutput(PipeOutput.pumpTo(outputStream)) <---- I need to set this

    .setProgressListener(new ProgressListener() {

        @Override

        public void onProgress(FFmpegProgress progress) {

            double percents = 100. * progress.getTimeMillis() / duration.get();

            System.out.println("Progress: " + percents + "%");

        }

    })

    .execute();

[Linus Larsen]

How about

// Create your resource

BlobId blobId = BlobId.of(YOUR_BUCKET, YOUR_FILENAME);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("video/mp4").build();
Blob blob = getStorage().create(blobInfo);

WriteChannel writeChannel = blob.writer();
// Then use writeChannel to write your bytes, it extends WritableByteChannel which has

public int write(ByteBuffer src)

I.e you have to create an OutputStream that writes to a ByteBuffer first.

https://gist.github.com/hoijui/7fe8a6d31b20ae7af945

Sort of.

/ Linus

[Richard Arnold]

Thanks for the reply. So is this going to overwrite the file specified? Or is it going to create a new file in memory that I then have to save to cloud storage myself? Could you show me how the code you wrote fits in with the code I have?

[Linus Larsen]

Well, I can't write all code for you. Basically I would not advice you to read and write potentially large video data files into memory. You will probably

run out of memory. Lets walk thru your code:

see Channels java doc: https://docs.oracle.com/javase/8/docs/api/java/nio/channels/Channels.html#newInputStream-java.nio.channels.ReadableByteChannel-

// This is the input file

BlobId id = BlobId.of("storageBucket", "videos/video_1.mp4");

byte[] content = storage.readAllBytes(id);

InputStream inputStream = new ByteArrayInputStream(content);

// You can probably use this instead to avoid reading everything in memory

Blob input = storage.get(id);

ReadChannel readChannel = input.reader();

InputStream inputStream = Channels.newInputStream(readChanne);

// Create output, write to a separate file then rename it to the original instead of overwriting input with output

BlobId blobId = BlobId.of("storageBucket", "videos/video_1_retrancoded.mp4");

BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("video/mp4").build();

Blob output = storage.create(blobInfo);

WriteChannel writeChannel = output.writer();

OutputStream outputStream = Channels.newOutputStream(writeChannel);

FFmpeg.atPath()

    .addInput(PipeInput.pumpFrom(inputStream))

    .setOverwriteOutput(true) // Dont see why you would need this?

    .addArguments("-movflags", "faststart")

    .addOutput(PipeOutput.pumpTo(outputStream)) <---- I need to set this

    .setProgressListener(new ProgressListener() {

        @Override

        public void onProgress(FFmpegProgress progress) {

            double percents = 100. * progress.getTimeMillis() / duration.get();

            System.out.println("Progress: " + percents + "%");

        }

    })

    .execute();

// When done rename output file to input file on google storage (or maybe it's a move you need to do.)

I just provided you with some pseudo code, you need to do the implementation, error handling etc. Also there are a bunch of

parameters that you can set on the readable / writable channels, you'll figure it out.

/ Linus

[George (Cloud Platform Support)]

Hello Richard, 

Maybe worth mentioning: the present discussion group is for general ideas on the Google App Engine, and related problems and trends. For coding in Java, you’ll be at an advantage to rather post your questions on Stackoverflow, to gain this way access to a large number of experts; it is meant for providing you help with coding.