cpu capabilities none!

[Reinis Nartiss]

Hallo, community!

I am using Matterhorn 1.4.3 and my issue is really long encoding time. some 1.5h long video preperation for Trim and review takes ~10 hours and finalization after trimming also more then 10 hours...

I have VM with 4cores of i7 which all are running those hours on almost full load. In Matterhorn log I see such input:

[libx264 @ 0x1aa12c0] using cpu capabilities: none!

I checked/googled and it says that ffmpeg needs to be compiled with yasm to enable those CPU capabilities, I think I did that and in libraries I have yasm installed. Does anyone have such problems and what are Your typical encoding time for videos. right now it takes 2 days to make video, which is obvious too much. 

PS it was same with default settings and a bit longer with HD encoding.

Please advice!

Thank You!

Reinis Nartiss

The University of Latvia

[Lars Kiesow]

Hi Reinis,
I don't have such problems, but it really depends on how you built
libx264 and ffmpeg. Did you built them yourself? Which version? Which
operating system/distribution? How did you configure them? Can you give
us the output of ffmpeg when you run it without arguments?

Additionally, it also depends on the configuration (encoding profiles)
you are using. E.g. using -preset placebo would slow a system down
immensely.

Regards,
Lars

[Greg Logan]

On 14-04-24 08:52 AM, Lars Kiesow wrote:
> Hi Reinis,
> I don't have such problems, but it really depends on how you built
> libx264 and ffmpeg. Did you built them yourself? Which version? Which
> operating system/distribution? How did you configure them? Can you give
> us the output of ffmpeg when you run it without arguments?
>
> Additionally, it also depends on the configuration (encoding profiles)
> you are using. E.g. using -preset placebo would slow a system down
> immensely.

Lars is correct, but there is a further issue: You're seeing this on a
VM. The VM host may not expose those features to its children. I'm not
100% sure how to test for this, but I'm sure that the 3rd party tools
script uses yasm to build things which says to me that the virtual CPUs
you have are not set up to provide those features.

G

[Sven Stauber]

Hi Reinis,

To find out what CPU capabilities are provided by the virtual CPUs, execute the command cat /proc/cpuinfo on the VM (should work on most Linux distributions).

This command lists information about all processors and its capabilities. The interesting part of the output in your case are the flags. Look for flags that indicate vector processing capabilities (SIMD) as mmx, sse, sse2, sse*_* and avx.

The output of the command line ffmpeg -version could also provide hints.

Regards,

Sven

[Lars Kiesow]

If you build FFmpeg somewhere else, there is also a
“--enable-runtime-cpudetect” configuration flag.
—Lars

On Thu, 24 Apr 2014 09:43:57 -0700 (PDT)
Sven Stauber <sven.s...@switch.ch> wrote:

> Hi Reinis,
>
> To find out what CPU capabilities are provided by the virtual CPUs,

> execute the command *cat /proc/cpuinfo *on the VM (should work on

> most Linux distributions).
>
> This command lists information about all processors and its
> capabilities. The interesting part of the output in your case are the

> *flags*. Look for flags that indicate vector processing capabilities

> (SIMD) as mmx, sse, sse2, sse*_* and avx.
>

> The output of the command line* ffmpeg -version* could also provide
> hints.
>
> Regards,
> Sven
>
>

[Reinis Nartiss]

Hi!

Thank You for lots of replys :)

First here are my flags from

cat /proc/cpuinfo

flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse3                                                                                                                                                             6 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor la

So those capabilities should work, thats good news!

[mhorn@mhornt3 ~]# ffmpeg -version
ffmpeg version 2.1.4
built on Feb 28 2014 14:04:05 with gcc 4.4.6 (GCC) 20120305 (Red Hat 4.4.6-4)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --disable-static --enable-runtime-cpudetect --enable-gpl --enable-version3 --enable-postproc --enable-avfilter --enable-pthreads --enable-x11grab --enable-vdpau --disable-avisynth --enable-frei0r --enable-libopencv --enable-libdc1394 --enable-libvo-aacenc --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libsoxr --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-bzlib --enable-libass --enable-libdc1394 --enable-libfreetype --enable-openal --enable-libopus --enable-libpulse --enable-libv4l2 --disable-debug --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC' --disable-stripping
libavutil      52. 48.101 / 52. 48.101
libavcodec     55. 39.101 / 55. 39.101
libavformat    55. 19.104 / 55. 19.104
libavdevice    55.  5.100 / 55.  5.100
libavfilter     3. 90.100 /  3. 90.100
libswscale      2.  5.101 /  2.  5.101
libswresample   0. 17.104 /  0. 17.104
libpostproc    52.  3.100 / 52.  3.100

about yasm:
[mhorn@mhornt1 ~]# rpm -qa | grep yasm
yasm-1.2.0-1.el6.x86_64

I tried to compile ffmpeg using this manual: https://trac.ffmpeg.org/wiki/CentosCompilationGuide

But then I couldn't normaly run matterhorn, got a lot of confusing errors...

Right now my setup is created using repository from: http://lernfunk.de/matterhorn-repo/matterhorn.repo

Can I somehow recompile ffmpeg while Matterhorn is allready installed? Maybe some good manual to use?

Thank You!

Reinis

--
You received this message because you are subscribed to the Google Groups "Matterhorn Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to matterhorn-use...@opencast.org.
To post to this group, send email to matterho...@opencast.org.
Visit this group at http://groups.google.com/a/opencast.org/group/matterhorn-users/.

[Sven Stauber]

Hi Reinis,

A thing that is weird with your CPU flags is that MMX is not available (flag mmx is missing), but interestingly, other vector processing extensions are (see, sse2).

This is weird in means of no physical IA-32 CPU ever built has vector processing capabilities but does not support MMX. Even this is quite unlikely, it may be worth the effort to quickly validate that ffmpeg developers do not simply assume that "no mmx = no vector processing" (this is true for physical CPUs up to now), which would result in your problem.

You can achieve this be just downloading a pre-compiled ffmpeg binary that statically links libx264 as found on http://ffmpeg.gusari.org/static/. Copy the ffmpeg binary to any location on your VM and run it to see whether CPU capabilities are used (I quickly checked such a binary and it reported [libx264 @ 0xba36200] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2 AVX).

Regards,

Sven

[Lars Kiesow]

Hi Reinis,
it's not an error in the ffmpeg build but an error in the built of
libx264. I'm updating the version in the repository now.

Thanks for noticing this and maybe you can give some feedback after the
update.

Regards,
Lars

> > execute the command *cat /proc/cpuinfo *on the VM (should work on

> > most Linux distributions).
> >
> > This command lists information about all processors and its
> > capabilities. The interesting part of the output in your case are

> > the *flags*. Look for flags that indicate vector processing

> > capabilities (SIMD) as mmx, sse, sse2, sse*_* and avx.
> >

> > The output of the command line* ffmpeg -version* could also provide

[Lars Kiesow]

Hi Reinis,
please run
yum update
to get the new version of libx264 from the Matterhorn RPM repository. It
should fix this issue.

Regards,
Lars

On Fri, 25 Apr 2014 14:39:51 +0200
Lars Kiesow <lki...@uos.de> wrote:

> Hi Reinis,
> it's not an error in the ffmpeg build but an error in the built of
> libx264. I'm updating the version in the repository now.
>
> Thanks for noticing this and maybe you can give some feedback after
> the update.
>
> Regards,
> Lars
>
>
> On Fri, 25 Apr 2014 14:55:25 +0300
> Reinis Nartiss <reinis....@gmail.com> wrote:
>

> > > CPUs, execute the command *cat /proc/cpuinfo *on the VM (should

> > > work on most Linux distributions).
> > >
> > > This command lists information about all processors and its
> > > capabilities. The interesting part of the output in your case are

> > > the *flags*. Look for flags that indicate vector processing

> > > capabilities (SIMD) as mmx, sse, sse2, sse*_* and avx.
> > >

> > > The output of the command line* ffmpeg -version* could also

[Reinis Nartiss]

Thank you, Lars!

It works!

[libx264 @ 0x1024ee0] using cpu capabilities: MMX2 SSE Cache64

Thank you all others too.

Regards,
Reinis

[Reza Toghraee]

hi reinis

did you experience any matterhorn Failed messages for recordings after upgrading your ffmpeg?

we are running ffmpeg 0.8 and sometimes the encoding process of ffmpeg fails. 

im thinking to upgrade the ffmpeg to a higher version. 

whats your recommendations? 

[Lars Kiesow]

Hi Reza,
I would definitely recommend to upgrade FFmpeg.
The Matterhorn RPM repositry ships FFmpeg 2.1.3 by default and 2.2.1
for testing (though 2.2.1 is definitely stable). I'm always using the
latest stable FFmpeg version and never experienced any problems in
regards to Matterhorn. Quite the contrary FFmpeg has grows faster, more
stable and comes with a lot of cool new features.

For Matterhorn 1.5, the default FFmpeg version will be upgraded as
well. I think that the version the install scripts now build is 2.1.2.
There are also new encoding profiles for the new FFmpeg version in the
1.5 branch but its almost completely a command line clean-up. The only
necessary change is documented here:
https://opencast.jira.com/wiki/display/MHDOC/Using+a+new+FFmpeg

Regards,
Lars

On Sat, 26 Apr 2014 13:55:49 +0000
Reza Toghraee <reza.t...@visionaire.com> wrote:

> hi reinis
>
> did you experience any matterhorn Failed messages for recordings
> after upgrading your ffmpeg? we are running ffmpeg 0.8 and sometimes
> the encoding process of ffmpeg fails. im thinking to upgrade the
> ffmpeg to a higher version. whats your recommendations?
>
>
>
> -------- Original message --------
> From: Reinis Nartiss <reinis....@gmail.com>
> Date: 26/04/2014 17:04 (GMT+04:00)
> To: matterho...@opencast.org
> Subject: Fwd: [Matterhorn-Users] Re: cpu capabilities none!
>
>
>
> Thank you, Lars!
>
> It works!
>
> [libx264 @ 0x1024ee0] using cpu capabilities: MMX2 SSE Cache64
>
> Thank you all others too.
>
> Regards,
> Reinis
>
>
> 2014-04-25 16:11 GMT+03:00 Lars Kiesow

> <lki...@uos.de<mailto:lki...@uos.de>>:

>
> Hi Reinis,
> please run
> yum update
> to get the new version of libx264 from the Matterhorn RPM repository.
> It should fix this issue.
>
> Regards,
> Lars
>
>
> On Fri, 25 Apr 2014 14:39:51 +0200
> Lars Kiesow <lki...@uos.de<mailto:lki...@uos.de>> wrote:
>
> > Hi Reinis,
> > it's not an error in the ffmpeg build but an error in the built of
> > libx264. I'm updating the version in the repository now.
> >
> > Thanks for noticing this and maybe you can give some feedback after
> > the update.
> >
> > Regards,
> > Lars
> >
> >
> > On Fri, 25 Apr 2014 14:55:25 +0300
> > Reinis Nartiss

> > > <sven.s...@switch.ch<mailto:sven.s...@switch.ch>>: