Using Composite Workflow

[Max Lira]

Hi

I'm trying to use the composite workflow to use later in the youtube integration.

Here is my code of the workflow (Based upon in the exaple materials:

 <operation
     id="composite"
     if="${publishToYouTube}"
     fail-on-error="true"
     exception-handler-workflow="error"
     description="Puts several input videos on a single canvas for Youtube publication">
     <configurations>
        <configuration key="source-flavor-upper">presentation/trimmed</configuration>
        <configuration key="source-flavor-lower">presenter/trimmed</configuration>
        <configuration key="source-tags-upper">comp,rss</configuration>
        <configuration key="source-tags-lower">comp,rss</configuration>
        <!--<configuration key="source-tags-watermark">branding</configuration>
        <configuration key="source-flavor-watermark">image/work</configuration>
        <configuration key="source-url-watermark">file:///Users/me/logo.jpg</configuration> -->
        <configuration key="encoding-profile">composite.http</configuration>
        <configuration key="target-tags">composite,rss,atom,archive</configuration>
        <configuration key="target-flavor">composite/delivery</configuration>
        <configuration key="output-resolution">1920x1080</configuration>
        <configuration key="output-background">black</configuration>
        <configuration key="layout">topleft</configuration>
        <configuration key="layout-topleft">
           {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};
           {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":0.0,"top":0.0}}};
           {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":1.0,"top":0.0}}}
        </configuration>
        <configuration key="layout-topright">
           {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":1.0}}};
           {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":1.0,"top":0.0}}};
           {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":20,"x":20},"reference":{"left":0.0,"top":0.0}}}
        </configuration>
     </configurations>
  </operation>

The problem is when I run it ffmpeg gave me this error (from opencast log)

(FFmpegEncoderEngine:189) - Invalid file index 1 in filtergraph description [0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower];[1:v]scale=384:216[upper];[lower] [upper]overlay=0:0[out];[0:a][1:a]amix=inputs=2[aout].

The source videos have this resolution input:

Presenter: 640x360
Presentation: 800x450

Presentation is the upper video, presenter is the lower one.

Opencast gave me this error in the administration panel:

The upper video 'http://somewhere.com/archive/archive/mediapackage/b7459b0b-479f-491e-97ad-370bd5869bf5/061806ef-1d2a-484a-ac51-4725c6b96db5/2/track.mp4' and lower video 'http://somewhere.com/archive/archive/mediapackage/b7459b0b-479f-491e-97ad-370bd5869bf5/4afe9afa-4c0f-4e39-8e4d-40e5ff2842aa/2/track.mp4' with the optional watermark '#{watermark}' could not be composite by the encoding profile 'composite.http' with the properties '{compositeCommand=-i /srv/opencast/workspace/http_classture.pucv.cl/archive/archive/mediapackage/b7459b0b-479f-491e-97ad-370bd5869bf5/061806ef-1d2a-484a-ac51-4725c6b96db5/2/track.mp4 -filter_complex [0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower];[1:v]scale=384:216[upper];[lower][upper]overlay=0:0[out];[0:a][1:a]amix=inputs=2[aout] -map [out] -map [aout]}'.

Can anyone help me?

Regards

[Lars Kiesow]

Hi Max,
the problem is that the filterchain uses 2 inputs but there is only one
input passed to ffmpeg. That is why FFmpeg is complaining.

Do both videos (presentation/trimmed and presenter/trimmed) exist in
your workflow instance?

Btb, if it's an ffmpeg problem, please always post the whole ffmpeg
related log, especially the command line used.

Regards,
Lars

[Sven Stauber]

Hi Max,

Probably your layout (although it seems to be a copy of the documentation) is wrong:

           {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"offset":{"y":-20,"x":-20},"reference":{"left":0.0,"top":0.0}}};

So you are taking the top-left corner as reference ("reference":{"left":0.0,"top":0.0}) and then you would place the video with an offset of -20/-20 ("offset":{"y":-20,"x":-20}) which is out of the canvas.

Try to set those offsets to zero to test whether my guess is right.

Best,

Sven

[Lars Kiesow]

Hi Sven,
would that cause the composite woh to not include one input file in the
FFmpeg command line? If so, then it sounds definitely like a bug. If it
already detects the mistake, it should complain and not build a
mismatching filter chain.
Regards,
Lars

[Ruth Lang]

Hi, 

the composite who is skipped, if there is only one video.

Regards

Ruth

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

_______________________________

Universität zu Köln

Regionales Rechenzentrum (RRZK)
Weyertal 121, Raum 4.08
D-50931 Köln
✆:  +49-221-470-89618

rrzk.uni-koeln.de

facebook.com/rrzkoeln

[Sven Stauber]

Hi Lars,

Hi Sven,
would that cause the composite woh to not include one input file in the
FFmpeg command line? If so, then it sounds definitely like a bug. If it
already detects the mistake, it should complain and not build a
mismatching filter chain.

No, it wouldn't. But as far as I know, WOH composite expects the streams to fit onto the canvas.

Best,

Sven

[Max Lira]

Thanks for your Answers guys, Im still with the problem, but I have some findings.

- The WOH is using the two video streams (presenter and presentation) and changed from trimmed to themed (like the compose WOH).

- I commented the source-tag-flavour and the watermark configuration keys, still with the problem.

-Trying to manually with ffmpeg to make this (using the parameters Opencast is using gave me this error:

[root@abrams-mk3 mlira]#
 ffmpeg -nostats -i
/srv/opencast/workspace/somewhere/archive/archive/mediapackage-4c0f-4e39-8e4d-40e5ff2842aa/2/track.mp4
 -filter:v -i
/srv/opencast/workspace/somewhere/archive/archive/me5/061806ef-1d2a-484a-ac51-4725c6b96db5/2/track.mp4
 -filter_complex
'[0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower]ay=0:0[out];[0:a][1:a]amix=inputs=2[aout]'
 -map [out] -map [aout] -c:a flac -c:v libx264 -crf 10 -preset fast
/srv/openarchive/mediapackage/b7459b0b-479f-491e-97ad-370bd5869bf5/4afe9afa-4c0f-4e39-8e4d-40e5ff2842aa/2/track_c8f81b00-1521-40
ffmpeg version 3.2.2-static http://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.1 (Debian 5.4.1-4) 20161202
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio tconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libass --enable-libfreetype --enable-libfribidi --enablee-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsoxr --enable-libspeex --enable-mrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0,
 mov,mp4,m4a,3gp,3g2,mj2, from
'/srv/opencast/workspace/somewhere/archive/archive/mediapackage/b70f-4e39-8e4d-40e5ff2842aa/2/track.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.25.101
  Duration: 01:16:03.44, start: 0.000000, bitrate: 290 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 189 lt)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Invalid file index 1 in filtergraph description [0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower];[1:v]scale=384:216[a]amix=inputs=2[aout].

- Testing again if ffmpeg have problems using the code in this page to make PIP, worked without problems.

ffmpeg -i pipInput2.flv -i pipInput1.flv -filter_complex "[1]scale=iw/5:ih/5 [pip]; [0][pip] overlay=main_w-overlay_w-10:10" -profile:v main -level 3.1 -b:v 440k -ar 44100 -ab 128k -s 720x400 -vcodec h264 -acodec libfaac PIP_output3.mp4

-Trying with the Opencast videos but using the console commands from above without problems I get the PIP video without problems

- Sven: I tried changing what you say but the problem persist.

With all of this tests I can asuume that the line:

0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower];[1:v]scale=384:216[a]amix=inputs=2[aout]

In ffmpeg have problems, so, can someone with this WOH to helpme with the JSON parameters?

[Lars Kiesow]

Hi Max,
the FFmpeg command you posted does make no sense:

ffmpeg -i ...mp4 -filter:v -i ...mp4 -filter_complex ...

The second -i would be interpreted as filter.

Since we are at the moment guissing wildly because we lack some
information, can you

…post the encoding profile you are using
…post the workflow you are using

and most importantly

…post the opencast.log part where the encoding failed (be generous
and better take a few more lines but most importantly, it should
include the FFmpeg command line)

Best use a site like http://fpaste.org for posting the data.

Regards,
Lars

> --enable-libvorbis -- enable-libvpx --enable-libwebp --enable-libx264

> <http://www.oodlestechnologies.com/blogs/PICTURE-IN-PICTURE-effect-using-FFMPEG>to

> make PIP, worked without problems.
>
> ffmpeg -i pipInput2.flv -i pipInput1.flv -filter_complex
> "[1]scale=iw/5:ih/5 [pip]; [0][pip] overlay=main_w-overlay_w-10:10"

> -profile:v main -level 3.1 - b:v 440k -ar 44100 -ab 128k -s 720x400

[Max Lira]

Hi Lars, thanks for your help.

Yep, that ffmpeg command don't make any sense, anyway, here is the log were you can see what opencast is running, this is only from the part when the job is started until its finished. https://da.gd/uXYn

The workflws, there are two workflows, the workflow to publish (ng-publish.xml) and the workflow who make the video processing (ng-partial-publish.xml), in the second one is the composite WOH.


The codec scheme, This is the default for composite installed in Opencast "opencast-movies.properties"

[Stefanos G]

Hi Max,

I was in your position 2 years ago. Although ffmpeg cmds actually worked
unfortunately OC uses json values to put the two streams.

I will give you an example from our videos.

We have a coverimage in png in 1280x720

      <operation
        id="composite"
        fail-on-error="true"
        exception-handler-workflow="ng-partial-error"
        description="Add coverimage to the combined video">
        <configurations>
          <configuration
key="source-flavor-upper">presentation/resized</configuration>
          <configuration
key="source-flavor-lower">presenter/trimmed</configuration>
          <configuration
key="source-tags-watermark">branding</configuration>
          <configuration
key="source-flavor-watermark">image/cover</configuration>
          <configuration
key="encoding-profile">composite.http</configuration>
          <configuration
key="target-flavor">composite/delivery</configuration>
          <configuration key="output-resolution">1280x720</configuration>
          <configuration key="output-background">blue</configuration>
          <configuration key="layout">topleft</configuration>
          <configuration key="layout-topleft">

{"horizontalCoverage":0.375,"anchorOffset":{"referring":{"left":0.0,"top":0.133},"reference":{"left":0.0,"top":0.133},"offset":{"x":0,"y":0}}};

{"horizontalCoverage":0.625,"anchorOffset":{"referring":{"left":1.0,"top":0.5},"reference":{"left":1.0,"top":0.5},"offset":{"x":0,"y":0}}};

{"horizontalCoverage":1.000,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"reference":{"left":1.0,"top":0.0},"offset":{"x":0,"y":0}}};
          </configuration>
          <configuration key="layout-topright">

{"horizontalCoverage":0.375,"anchorOffset":{"referring":{"left":0.0,"top":0.133},"reference":{"left":0.0,"top":0.133},"offset":{"x":0,"y":0}}};

{"horizontalCoverage":0.625,"anchorOffset":{"referring":{"left":1.0,"top":0.5},"reference":{"left":1.0,"top":0.5},"offset":{"x":0,"y":0}}};

{"horizontalCoverage":1.000,"anchorOffset":{"referring":{"left":1.0,"top":0.0},"reference":{"left":1.0,"top":0.0},"offset":{"x":0,"y":0}}};
          </configuration>
        </configurations>
  </operation>

This is our step. In the left side we have a presenter video with
480x330 and in the right a presentation with 800x600. Padding top 30px,
padding bottom 30px(as you can see from the png).  Sven explained to me
at that time how this WOH works. Unfortunately I cannot find the
email/thread any more.

But in general you have to do the math. In our case I remember only

{"horizontalCoverage":0.375 -> 480/1280
{"horizontalCoverage":0.625 -> 800/1280
{"horizontalCoverage":1.000 -> 1280/1280

The another values came also with the same way (y divided with total
length) and after a lot of testing.
Sotake some painkillers  and start doing the math for you case.

Hope my email helps you a bit.

Regards
Stefanos
On 02/09/2017 11:47 PM, Max Lira wrote:

 for posting the data.

    Regards,
    Lars


    On Thu, 9 Feb 2017 13:16:46 -0800 (PST)
    Max Lira <max...@gmail.com <javascript:>> wrote:

    > Thanks for your Answers guys, Im still with the problem, but I have
    > some findings.
    >
    > - The WOH is using the two video streams (presenter and presentation)
    > and changed from trimmed to themed (like the compose WOH).
    >
    > - I commented the source-tag-flavour and the watermark configuration
    > keys, still with the problem.
    >
    > -Trying to manually with ffmpeg to make this (using the parameters
    > Opencast is using gave me this error:
    >
    > [root@abrams-mk3 mlira]#
    >  ffmpeg -nostats -i
    >
    /srv/opencast/workspace/somewhere/archive/archive/mediapackage-4c0f-4e39-

    > 8e4d-40e5ff2842aa/2/track.mp4
    >  -filter:v -i
    >
    /srv/opencast/workspace/somewhere/archive/archive/me5/061806ef-1d2a-484a-

    > ac51-4725c6b96db5/2/track.mp4
    >  -filter_complex
    >
    '[0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower]ay=0:0[out];[0:a][1:a]amix=inputs=2[aout]'

    >  -map [out] -map [aout] -c:a flac -c:v libx264 -crf 10 -preset fast
    >
    /srv/openarchive/mediapackage/b7459b0b-479f-491e-97ad-370bd5869bf5/4afe9afa-

    > 4c0f-4e39-8e4d-40e5ff2842aa/2/track_c8f81b00-1521-40
    > ffmpeg version 3.2.2-static http://johnvansickle.com/ffmpeg/

>to

    > make PIP, worked without problems.
    >
    > ffmpeg -i pipInput2.flv -i pipInput1.flv -filter_complex
    > "[1]scale=iw/5:ih/5 [pip]; [0][pip] overlay=main_w-overlay_w-10:10"
    > -profile:v main -level 3.1 - b:v 440k -ar 44100 -ab 128k -s 720x400
    > -vcodec h264 -acodec libfaac PIP_output3.mp4
    >
    >
    >
    > -Trying with the Opencast videos but using the console commands from
    > above without problems I get the PIP video without problems
    >
    > - Sven: I tried changing what you say but the problem persist.
    >
    > With all of this tests I can asuume that the line:
    >
    0:v]scale=1920:1080,pad=1920:1080:0:0:black[lower];[1:v]scale=384:216[a]amix

    > =inputs=2[aout]
    >
    > In ffmpeg have problems, so, can someone with this WOH to helpme with
    > the JSON parameters?
    >
    > El jueves, 9 de febrero de 2017, 16:56:44 (UTC-3), Sven Stauber
    > escribió:
    > >
    > > Hi Lars,
    > >
    > > Hi Sven,  
    > >> would that cause the composite woh to not include one input file
    > >> in the FFmpeg command line? If so, then it sounds definitely like
    > >> a bug. If it already detects the mistake, it should complain and
    > >> not build a mismatching filter chain.
    > >>  
    > >
    > > No, it wouldn't. But as far as I know, WOH composite expects the
    > > streams to fit onto the canvas.
    > >
    > > Best,
    > > Sven
    > >  
    >

-- 
You received this message because you are subscribed to the Google
Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to users+un...@opencast.org
<mailto:users+un...@opencast.org>.

TechFakpsd.png

[Corné]

Hi Max,

Could you please try to run this composite step (it's a bit simplified - but it works for our workflow)

        <operation 

          id="composite" 

          if="${publishToYouTube}" 

          fail-on-error="true" 

          exception-handler-workflow="error" 

          description="Puts several input videos on a single canvas for Youtube publication">

          <configurations>

            <configuration key="source-flavor-upper">presentation/themed</configuration>

            <configuration key="source-flavor-lower">presenter/themed</configuration>

            <configuration key="encoding-profile">composite.http</configuration>

            <configuration key="target-tags">composite,rss,atom,archive</configuration>

            <configuration key="target-flavor">composite/delivery</configuration>

            <configuration key="output-resolution">1920x1080</configuration>

            <configuration key="output-background">0x000000FF</configuration>

            <configuration key="layout">topleft</configuration>

            <configuration key="layout-single">

              {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"reference":{"left":0.0,"top":0.0},"offset":{"x":0,"y":0}}};

            </configuration>

            <configuration key="layout-topleft">

              {"horizontalCoverage":1.0,"anchorOffset":{"referring":{"left":0.0,"top":0.0},"reference":{"left":0.0,"top":0.0},"offset":{"x":0,"y":0}}};

              {"horizontalCoverage":0.2,"anchorOffset":{"referring":{"left":1.0,"top":1.0},"reference":{"left":1.0,"top":0.0},"offset":{"x":0,"y":0}}};

            </configuration>

          </configurations>

        </operation>

Kind regards,
Corne

[Sven Stauber]

Hi Max,

If Opencast triggers that ffmpeg command:

[root@abrams-mk3 mlira]#
 ffmpeg -nostats -i 

/srv/opencast/workspace/somewhere/archive/archive/mediapackage-4c0f-4e39-8e4d-40e5ff2842aa/2/track.mp4
 -filter:v -i 
/srv/opencast/workspace/somewhere/archive/archive/me5/061806ef-1d2a-484a-ac51-4725c6b96db5/2/track.mp4
 -filter_complex 

... then you have an error in the encoding profile you are using (the line -filter:v definitely should not be there and probably comes from the encoding profile you use).

IIRC, there was an error in the encoding profile in some earlier 2.x versions of Opencast (which version of Opencast are you using, btw?).

Have a look at https://bitbucket.org/opencast-community/matterhorn/src/fc413dde976bb947812754ccd273a0d5ef1cca7c/etc/encoding/opencast-movies.properties?at=develop&fileviewer=file-view-default to see a correct encoding profile as for use with WOH composite.

The key thing here is that there must be exactly one #{compositeCommand} and no other filters in the encoding profile.

 

By the way: Note that WOH composite has been enhanced recently to improve PiP support. In the upcoming 2.4, you can take the video resolution of an input file to specify the canvas size (vs. a fixed resolution as now set by output-resolution).

Best,

Sven

[Max Lira]

Hi.

Thanks your four your great support, I tried your solutions and now Composite WOH is working very well.

The problem was the encoding profile that opencast 2.2.4 had, I changed with the version Sven posted.

Also, Corne, your Composite operation is very good. I tried and works!! Can I use that example to pull in the documentation?

Stephanos, thanks for the advice about and your experience, I will copy the idea to make a custom background, looks very good.

Now I have to do more tests to make the youtube publication using the composite WOH works.

Thanks to everyone

[Corné]

Hi Max,

Glad it worked :) Yes, please use it wherever you want.

Kind regards,
Corné