FFMPEG Parallel processing and workflow

[Hans Erasmus]

Hi all

We are trying to convert our old workflow and ffmpeg command to newer parallel processing commands.  But we seem to not understand something fully. When looking at the examples, please keep the following in mind:

We have 2 audio streams coming in from recorded videos, 1 presenter audio and 1 translator audio file. We normally create the "work" version of the files by muxing the presenter audio onto the camera.avi and the translator audio onto screen.avi, to create 2 mp4 files.

First up, here is the code for our old workflow, which work perfectly (and still does) in 1.5.1.

Workflow (only SOME of the excerpts):

<!-- Composite 720p -->

<operation

      id="composite"

      retry-strategy="retry"

      max-attempts="5"    

      fail-on-error="true"

      exception-handler-workflow="error"

      description="Encoding Composite 720p">

      <configurations>

        <configuration key="source-flavor-upper">presentation/trimmed</configuration>

        <configuration key="source-flavor-lower">presenter/trimmed</configuration>

        <configuration key="source-tags-upper"></configuration>

        <configuration key="source-tags-lower"></configuration> -->

        <!-- <configuration key="source-tags-watermark">branding</configuration>

        <configuration key="source-flavor-watermark">image/work</configuration>

        <configuration key="source-url-watermark">file:///Users/me/logo.jpg</configuration>

        <configuration key="encoding-profile">compositenwu720.http</configuration>

        <configuration key="target-tags">composite</configuration>

        <configuration key="target-flavor">composite/delivery</configuration>

        <configuration key="output-resolution">1900x1080</configuration>

        <configuration key="output-background">red</configuration>

        <configuration key="layout">topleft</configuration>

        <configuration key="layout-topleft">

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};

          {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":0.0,"top":0.0}}};

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":1.0,"top":0.0}}}

        </configuration>

        <configuration key="layout-topright">

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};

          {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":0.0}}};

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":0.0,"top":0.0}}}

        </configuration>

      </configurations>

    </operation>

        <!-- Composite 720p -->

And its corresponding ffmpeg:

# compositenwu720

#   This profile is used for the composite workflow operation which puts

#   several input videos on a single canvas. The new video is put into an MP4

#   container and encoded using libfdk_aac and libx264 with high quality settings.

#   You can change the value of -preset (i.e. faster) if you want this to be

#   faster (it will cost you space) and the quality (-q:a and -crf).

profile.compositenwu720.http.name = 720p Side by Side Composite

profile.compositenwu720.http.input = visual

profile.compositenwu720.http.output = visual

profile.compositenwu720.http.suffix = -compound.mp4

profile.compositenwu720.http.mimetype = video/mp4

profile.compositenwu720.http.ffmpeg.command = -i #{in.audio.path} -i #{in.video.path} -map 1:0 -map 1:1 -map 0:0 -map 0:1 -filter_complex [0:v]scale=1280:720,setpts=PTS-STARTPTS,pad=iw*2:ih[bg];[1:v]scale=1280:720,setpts=PTS-STARTPTS[fg];[bg][fg]overlay=w #{out.dir}/#{out.name}#{out.suffix}

As stated this works well. More of these workflow profiles exist for all the steps that follow, but are just copies with other mappings, so I am not going to paste it.

Our attempt at parallel processing:

Workflow:

<!-- Parallel NWU 720p -->

    <operation

      id="composite"

      retry-strategy="retry"

      max-attempts="5"    

      fail-on-error="true"

      exception-handler-workflow="error"

      description="Parallel Encoding 720p">

      <configurations>

        <configuration key="source-flavor-upper">presentation/trimmed</configuration>

        <configuration key="source-flavor-lower">presenter/trimmed</configuration>

        <configuration key="source-tags-upper"></configuration>

        <configuration key="source-tags-lower"></configuration>

        <!-- <configuration key="source-tags-watermark">branding</configuration>

                     <configuration key="source-flavor-watermark">image/work</configuration>

        <configuration key="source-url-watermark">file:///Users/me/logo.jpg</configuration> -->

        <configuration key="encoding-profile">parallelnwu720.http</configuration>

        <configuration key="target-tags">composite</configuration>

        <configuration key="target-flavor">composite/delivery</configuration>

        <configuration key="output-resolution">1900x1080</configuration>

        <configuration key="output-background">red</configuration>

        <configuration key="layout">topleft</configuration>

        <configuration key="layout-topleft">

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};

          {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":0.0,"top":0.0}}};

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":1.0,"top":0.0}}}

        </configuration>

        <configuration key="layout-topright">

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};

          {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":0.0}}};

          {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":0.0,"top":0.0}}}

        </configuration>

      </configurations>

    </operation>

        <!-- Parallel NWU 720p -->

Please note:  We have tried changing id="composite" to id="encode", but it didn't work. 

Log line for that error: 2016-10-19 15:57:37,038 | INFO  | (EncodeWorkflowOperationHandler:166) - No source tags or flavors have been specified, not matching anything

FFMPEG profile:

#Parallel 720p NWU Encoding profile

profile.parallelnwu720.http.name = 720p parallel video encoding

profile.parallelnwu720.http.input = visual

profile.parallelnwu720.http.output = visual

profile.parallelnwu720.http.suffix.720comp-ver = -720comp.mp4

profile.parallelnwu720.http.suffix.720present-ver = -720present.mp4

profile.parallelnwu720.http.suffix.720ppt-ver = -720ppt.mp4

profile.parallelnwu720.http.suffix.720present2-ver = -720present2.mp4

profile.parallelnwu720.http.ffmpeg.command = -i #{in.audio.path} -i #{in.video.path} \

-map 1:0 -map 1:1 -map 0:0 -map 0:1 -filter_complex [0:v]scale=1280:720,setpts=PTS-STARTPTS,pad=iw*2:ih[bg];[1:v]scale=1280:720,setpts=PTS-STARTPTS[fg];[bg][fg]overlay=w #{out.dir}/#{out.name}#{out.suffix.720comp-ver} \

-map 1:1 -map 0:1 -filter_complex [1:v]scale=1280:720 -vcodec libx264 -r 25 -preset medium -crf 22 -qscale 3 -b:v 2500k -acodec libfdk_aac -aq 64 -ar 44100 -b:a 384k -movflags faststart -f mp4 -threads 0 #{out.dir}/#{out.name}#{out.suffix.720present-ver} \

-map 0:1 -map 1:1 -filter_complex [0:v]scale=1280:720 -vcodec libx264 -r 25 -preset medium -crf 22 -qscale 3 -b:v 2500k -acodec libfdk_aac -aq 64 -ar 44100 -b:a 384k -movflags faststart -f mp4 -threads 0 #{out.dir}/#{out.name}#{out.suffix.720ppt-ver} \

-map 0:1 -map 1:1 -filter_complex [1:v]scale=1280:720 -vcodec libx264 -r 25 -preset medium -crf 22 -qscale 3 -b:v 2500k -acodec libfdk_aac -aq 64 -ar 44100 -b:a 384k -movflags faststart -f mp4 -threads 0 #{out.dir}/#{out.name}#{out.suffix.720present2-ver}

So we are hoping anyone here has an idea. Hopefully we haven't gone totally crazy thinking this might actually work.

Regards

Hans

North-West University South Africa

[Rüdiger Rolf]

Hi Hans,

I have my doubts that the encode operation has an audio.in.path variable. The operation only takes one input file.

Regards
Rüdiger

Am 20.10.16 um 08:00 schrieb Hans Erasmus:

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

-- 
________________________________________________
Rüdiger Rolf, M.A.
Universität Osnabrück - Zentrum virtUOS
Heger-Tor-Wall 12, 49069 Osnabrück
Telefon: (0541) 969-6511 - Fax: (0541) 969-16511
E-Mail: rr...@uni-osnabrueck.de
Internet: www.virtuos.uni-osnabrueck.de

[Sven Stauber]

Hi Hans,

The encoding profile compositenwu720 doesn't make too much sense with WOH composite since WOH composite will calculate a complex filter based on the layout settings and add that complex filter to the encoding profile by replacing the variable #{compositeCommand} found in the encoding profile.

Well, if this works for you, you can add the ffmpeg -threads 6 directly to the encoding profile compositenwu720 to allow ffmpeg to use more cores (here: up to 6).

If you want to use WOH encode (for the parallel style encoding profiles), you would need to set the configuration keys correctly (see https://docs.opencast.org/r/2.3.x/admin/workflowoperationhandlers/encode-woh/).

Not sure how #{in.audo.path} is dealt with: Finally, the WOH would be required to replace this variable with some actual values....

Best,

Sven