one upload, multiple versions, one version is actually a collection of many files...

[Giovanni Cangiani]

Dear all,

I have a small rails application where I upload video files (typically 1 hour in duration, 1 to 2 Gb in size). For playing the videos, I use 

mediaelementjs which supports the HLS format that is just a way to segment the full video into small chunks for emulating video streaming as a sequence of small standard http downloads. The timing informations for the video chunks are served as a playlist (m3u8) file. Needless to say, the number of chunks is not constant but depends on the input video (length, bitrate, etc.).

Presently I use paperclip with a custom made processor for uploading and transcoding the video file into the various formats. From the point of view of Paperclip, the HLS file is the playlist but, in reality, the transcoder creates a directory with the same base name as the HLS file which contains the video chunks which are referenced as relative paths wrt the HLS file. 

This approach only works with local filesystem as storage backend and breaks the "one version = one file" assumption of Paperclip: the HLS chunks are created at a given point of the standard Paperclip post-processing pipeline by a callback function meant for completely different purpose.

I would like to rewrite everything in a cleaner and more "standard" way that would enable the use of arbitrary storage backend. 

I thought it would be a good occasion for eventually switching to shrine, in particular if you think my use case might be better handled within shrine than within paperclip.

Suggestions on a clean way to implement this within shrine would be greatly appreciated.

Thank you very much in advance.

[Janko Marohnić]

Yep, sounds like what you're doing should be possible to hook up nicely with Shrine, in a better way than with Paperclip. With Shrine you can handle these video chunks as individual versions, and then they can be uploaded to any storage backend.

For example, let's say that you want to transcode the video to .mp4, and create the HLS playlist, this is how the uploader might look like:

class VideoUploader < Shrine

  plugin :processing

  plugin :versions

  plugin :delete_raw # delete processed chunks after they're uploaded

  process(:store) do |io, context|

    raw_video = io.download

    mp4 = Tempfile.new(["video", ".mp4"], binmode: true)

    movie = FFMPEG::Movie.new(raw_video.path)

    movie.transcode(mp4.path)

    raw_video.delete # delete the downloaded raw video

    hls_path, chunk_paths = generate_hls(mp4) # custom method

    versions = {hls: File.open(hls_path)}

    chunk_paths.each_with_index do |path, idx|

      versions[:"#{chunk}_#{idx}"] = File.open(path)

    end

    versions # hash of files that will be uploaded

  end

  def generate_location(io, context)

    case context[:version]

    when :hls

      # return location where this HLS playlist should be uploaded to

    when /chunk_\d+/

      # return location where this chunk should be uploaded to

    else

      super

    end

  end

end

This is of course just based on assumptions I had from your description, but I think your use case should be able to fit in this frame.

Since your app is accepting large file uploads, you can also implement resumable file uploads from the browser using tus-ruby-server, you can see the demo app for how you can use it with Shrine. This is of course unrelated to processing, it's just another benefit of switching to Shrine.

Kind regards,

Janko

--
You received this message because you are subscribed to the Google Groups "Shrine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine+unsubscribe@googlegroups.com.
To post to this group, send email to ruby-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-shrine/3b0d2297-5510-4612-acfa-a7fe9dcc946d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Giovanni Cangiani]

That's great Janko! I really appreciate your fast and super-useful reply.

I am going to try and let you know.

Thank you.

To unsubscribe from this group and stop receiving emails from it, send an email to ruby-shrine...@googlegroups.com.