Video file normalization failures

[Mark Smith]

I am processing a small set of video files (in various formats) and am encountering an error message for three of them.  The directory contains one .mov, one .wmv, one MPEG-2 and two MPEG-4.  Normalization by FFMPEG works fine for the .mov and one of the MPEG-4 files, but fails for the other three.  Does anyone have any idea what I might be doing wrong?

[Kari R Smith]

Hi Mark,

Have you checked the FPR settings for those formats and the rules / instructions for what FFMPEG is doing?  I’d start there to diagnose the problems.

 

Kari

 

Kari R. Smith

Digital Archivist and Program Head for Born-digital Archives

Institute Archives and Special Collections

Massachusetts Institute of Technology Libraries, Cambridge, Massachusetts

617.253.5690   smithkr at mit.edu   http://libraries.mit.edu/archives/  @karirene69

--
You received this message because you are subscribed to the Google Groups "archivematica" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.
To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

[Mark Smith]

Thanks Kari. I'll check that out.

Mark

On 1 Jun 2017 16:02, "Kari R Smith" <smi...@mit.edu> wrote:

Hi Mark,

Have you checked the FPR settings for those formats and the rules / instructions for what FFMPEG is doing?  I’d start there to diagnose the problems.

 

Kari

 

Kari R. Smith

Digital Archivist and Program Head for Born-digital Archives

Institute Archives and Special Collections

Massachusetts Institute of Technology Libraries, Cambridge, Massachusetts

617.253.5690   smithkr at mit.edu   http://libraries.mit.edu/archives/  @karirene69

 

From: archiv...@googlegroups.com [mailto:archivematica@googlegroups.com] On Behalf Of Mark Smith
Sent: Thursday, June 01, 2017 9:33 AM
To: archivematica <archivematica@googlegroups.com>
Subject: [archivematica] Video file normalization failures

 

I am processing a small set of video files (in various formats) and am encountering an error message for three of them.  The directory contains one .mov, one .wmv, one MPEG-2 and two MPEG-4.  Normalization by FFMPEG works fine for the .mov and one of the MPEG-4 files, but fails for the other three.  Does anyone have any idea what I might be doing wrong?

--
You received this message because you are subscribed to the Google Groups "archivematica" group.

To unsubscribe from this group and stop receiving emails from it, send an email to archivematica+unsubscribe@googlegroups.com.

To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "archivematica" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/archivematica/qgLfRz-VRpw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to archivematica+unsubscribe@googlegroups.com.

[Andrew Berger]

Hi Mark,

Looking at the logs, I think you're running into two separate issues. In the first one, it looks like the FPR command for FFMPEG isn't able to handle the parentheses in the filename of the normalized file. If I'm understanding the command right, the input file path is quoted but the output file path is not, and that may be leading the shell to interpret the '(' in '(1)' as a special character rather than as part of the filename.

The other, possibly larger, issue is that it looks like Archivematica is running out of processing space, as the last two screenshots both have "no space left on device" errors. In my experience, WMV, MP4, and ProRes all get larger when you convert to MKV using the preservation normalization defaults in Archivematica. Transcoding to MKV using the ffv1 codec saves space when you're transcoding from uncompressed video, but when your source video is already highly compressed, it can often increase file size. My guess is that at least one of the successfully normalized files is substantially larger than its source, and that Archivematica is then running out of space while trying to transcode the remaining files. MOV can hold a variety of codecs, from uncompressed to highly compressed, so without knowing what kind of MOV that is, I can't say whether the normalized file is likely to be larger. The MP4 is almost certainly going to be larger as an MKV.

Andrew

On Thursday, June 1, 2017 at 8:03:14 AM UTC-7, Mark Smith wrote:

Thanks Kari. I'll check that out.

Mark

On 1 Jun 2017 16:02, "Kari R Smith" <smi...@mit.edu> wrote:

Hi Mark,

Have you checked the FPR settings for those formats and the rules / instructions for what FFMPEG is doing?  I’d start there to diagnose the problems.

 

Kari

 

Kari R. Smith

Digital Archivist and Program Head for Born-digital Archives

Institute Archives and Special Collections

Massachusetts Institute of Technology Libraries, Cambridge, Massachusetts

617.253.5690   smithkr at mit.edu   http://libraries.mit.edu/archives/  @karirene69

 

From: archiv...@googlegroups.com [mailto:archiv...@googlegroups.com] On Behalf Of Mark Smith
Sent: Thursday, June 01, 2017 9:33 AM
To: archivematica <archiv...@googlegroups.com>
Subject: [archivematica] Video file normalization failures

 

I am processing a small set of video files (in various formats) and am encountering an error message for three of them.  The directory contains one .mov, one .wmv, one MPEG-2 and two MPEG-4.  Normalization by FFMPEG works fine for the .mov and one of the MPEG-4 files, but fails for the other three.  Does anyone have any idea what I might be doing wrong?

--
You received this message because you are subscribed to the Google Groups "archivematica" group.

To unsubscribe from this group and stop receiving emails from it, send an email to archivematic...@googlegroups.com.

To post to this group, send email to archiv...@googlegroups.com.
Visit this group at https://groups.google.com/group/archivematica.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "archivematica" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/archivematica/qgLfRz-VRpw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to archivematic...@googlegroups.com.

[Mark Smith]

Hi Andrew

Many thanks for this.  Yes, the more I looked at this, the more I'd started to suspect the parentheses of causing a problem.  I've just renamed the file and processing it again has confirmed that suspicion.  It now, though, runs into the same space issue as the other files.  That will, I hope, be resolvable.  We have Archivematica installed for testing at the moment but, as I'm now sure we're going to use it long term, I'll mention this problem to our IT people when they come to do the permanent installation.  Handy to know about the trouble parentheses can cause.  Thanks for pointing that out.

Mark

[Timothy Walsh]

Hi Mark, all,

I ran into this same issue a while ago and the fix is quick/simple. In the Preservation Planning tab, go to the Normalization command "Transcoding to mkv with ffmpeg" and edit the following line:

command="${command} ${outputFile}"

All that's missing is to quote the ${outputFile} with single quotes or (to maintain the style convention) escaped double quotes, like so:

command="${command} \"${outputFile}\""

(you'll note this is the same treatment given to the input file earlier in the command)

That should work for you and doesn't require any tinkering in the code.

Cheers,
Tim

[Timothy Walsh]

Of course, Andrew already pointed this out as one of your problems, just thought some specific instructions for how to fix might be appreciated for anyone else who runs into this issue :)

[Kari R Smith]

+1

Appreciated, Tim!

 

Kari R. Smith

Digital Archivist and Program Head for Born-digital Archives

Institute Archives and Special Collections

Massachusetts Institute of Technology Libraries, Cambridge, Massachusetts

617.253.5690   smithkr at mit.edu   http://libraries.mit.edu/archives/  @karirene69

 

[Mark Smith]

Thanks Tim. I'll give that a go at work tomorrow.

Mark

To unsubscribe from this group and all its topics, send an email to archivematica+unsubscribe@googlegroups.com.