"Your system has run out of application memory"

[Alan Harvey]

Greetings,

I use QLab 3 to run video playlists on our Biology Department's Christie microtile display. Machine is a Mac Mini, system 10.9.2. Don't use the machine for anything else. Periodically, the program would freeze and I'd have to do the cmd-opt-shift-esc single-program forced quit, which yielded the error message in the subject title. A quick internet search shows this to be an unfortunately common and mysterious problem in Maverick and Yosemite, linked to a number of programs and/or processes. It happened rarely enough for me that I didn't worry about it too much.

However, for inexplicable reasons I'm seeing this on an increasing frequency, now up to on a daily basis. The nature of QLab is such that you can't use Activity Monitor to see what processes are operating at the time of the crash (by the time you force-quit out of QLab, activity levels have gone back to baseline levels). However, I did esc out of an active show, and discovered that the process VTdecoderXPCservice is working very, very hard, often well over 100% CPU, and suspect that this may be the problem. The only thread I've seen here that mentions VTdecoderXPCservice compares the performance of QLab 2 and QLab 3 with various codecs, and notes that this process dominates CPU use with QLab 3 and ProRes LT. As it so happens, I've been switching from H264 to ProRes LT, based on an article on the Figure 53 website.

So my questions are 1) has anyone experienced this particular problem using QLab 3? If so, have you found a solution? And what's the current thought about the stability of H264 vs. Apple ProRes LT on QLab 3?

Thanks for any insights here!

Cheers,

Alan

[Chris Ashworth]

Hi there Alan,

This sounds like a memory leak.  I’m aware of one memory leak in Apple’s code (which we are not able to fix) which happens when playing a video file that has an audio track. It’s a relatively small leak but would turn up for a long-looping video of the kind you describe here.

Is this possibly the culprit?

Best,

Chris

[Alan Harvey]

Chris,

It's possible, since, the playlist consists entirely of short videos, all of which have an audio track. None of the dozen or so videos in the playlist are terribly long (I think the longest one currently playing is less than six minutes, and some are less than two). What's troubling is that in the last week or two this has gone from a once-a-month issue to a daily issue; the only change I've made is to rely more on ProRes 422 LT and less on H264 whenever possible.

Any suggestions, even if kludgy, would be appreciated! The Apple forums are full of suggestions re: memory leaks, but none seem to be broadly successful.

Cheers,

Alan

[Andy Dolph]

I've been tending towards H.264 for most things (primarily because I'm given so much media already in that format) and I've been perfectly happy with it.

There may be situations where one codec or the other will relieve a bottleneck somewhere in the system and make things work better, but I think if H.264 has been working for you, I'd just stick with it unless there's some problem you are trying to solve by going to ProRes LT.

This is a total kluge but something that might be worth thinking about - what about an applescript or something similar that reboots the machine every night at midnight and a script that runs on login, loads qlab and starts the show?

If you can get the problem down to less then every 24 hrs, that might be a reasonable solution...

Andy

--
--
Change your preferences or unsubscribe here:
http://groups.google.com/group/qlab
 
Follow Figure 53 on Twitter: http://twitter.com/Figure53

---
You received this message because you are subscribed to the Google Groups "QLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alan Harvey]

Well, it went down twice today, which is pretty bad, but with that came an interesting observation. Namely, that although the videos that I create myself have all been ProRes 422, I do occasionally use videos from other sources, which are usually H264. However, the last one I added (a couple of weeks ago, significantly) was an mp4 (which I don't think I've ever used before). Not only that, but often (not always, but more than you'd expect by chance alone), that's the video that's on screen during the freeze. So I yanked it this morning after the second crash, and we'll see what happens.

That Figure 53 article ("Prepare, Execute, Troubleshoot") comparing ProRes 422 LT to H264 made a lot of sense to me (i.e., large files means less stress on your computer's processor), but some threads in this forum make me think that I need to do some additional testing. Certainly my life would be easier if I could reliably use the 90% smaller H264 format!

Cheers,

Alan

[Andy Dolph]

If pulling that file seems to help…

I might suggest that you try reencoding that .mp4 at the resolution and in the Codec you use for your files.

Sent from my iPhone

[Andy Lang]

Alan, another thing to check is to make sure that you don't have your
videos set to hold on the last frame. If you do, and don't explicitly
stop them when they are no longer visible, they'll keep running
invisibly in the background, and can rapidly pile up in memory. It's
super important to either let videos stop when they end, or stop them
with Stop or Fade cues (with their mode set to "Stop target when
done") after they're no longer in use.

Without seeing your workspace, I don't know if this is the case, but
figure it's worth mentioning to be certain!

-Andy

[Sam Kusnetz]

Alan Harvey wrote:
> That Figure 53 article ("Prepare, Execute, Troubleshoot") comparing
> ProRes 422 LT to H264 made a lot of sense to me (i.e., large files
> means less stress on your computer's processor), but some threads in
> this forum make me think that I need to do some additional testing.
> Certainly my life would be easier if I could reliably use the 90%
> smaller H264 format!

There are two other reasons I'm in favor of ProRes over H.264:

First, ProRes 422 HQ, 422, and 422 LT are pretty unequivocally better
looking than H.264.

Second, H.264 is a temporal codec, meaning that it encodes key frames
which contain a full screen worth of image, then a series of
intermediary frames which only encode data about what's changed since
the last keyframe. It's most common to have one keyframe per second, so
one out of every 24 frames in a 24 fps video. What that means
functionally is that rewinding and fast forwarding take a huge amount of
extra work, because unless you land on a keyframe, QLab needs to scan
backwards in time to the last keyframe, and then decode each frame up to
the frame you land on. Latency often ensues, even on fast hardware.

VTdecoderXPCservice is AVFoundation's video processing service, which
QLab calls to decode video. It will readily use available system
resources, as you've noted, but that doesn't indicate that there's
anything wrong. After all, that's why the processor is there, right? To
be used.

Finally, it's important to note the difference between a container
format and a codec when it comes to video. .mp4 video files are
containers which often use H.264 video, but not always. Sometimes they
can contain DivX or Xvid video, for example. So just looking at the
extension on the file name is not enough to tell you what's going on
inside the file. If you open the file with QuickTime Player and the show
the Move Inspector (command-I) you'll be able to find out what kind of
video is in your file.

One thing we have recently learned is that most of the time, h.264 high
profile video does not work well with QLab. You want to use baseline
profile or possibly either extended profile or main profile.

Cheerio
Sam

--
Sam Kusnetz | Figure 53 Field Operative
s...@figure53.com

[Alan Harvey]

Andy,

I've seen the occasional mention of this "hold last frame," but can't quite figure it out, and don't see anywhere in the Cue List workspace where it is mentioned or set. It's not something I've set for any video; is it a QLab parameter, or an attribute that can vary with incoming videos?

Alan

[Alan Harvey]

Sam,

Useful info here. I normally produce videos in PP, where I can export it any way I want; rarely have I just taken a video from elsewhere and popped it into QLab without messing with it, and re-exporting it, in PP. Live and learn!

Also interesting to learn that QLab has problems with H264 High Profile; PP (at least PP6) doesn't offer Extended, and Main and Baseline at least sound inferior to High, but not if they work and High doesn't! The (possibly) problematic mp4 file does use H264, but QT doesn't tell me which profile. The fact that this issue has happened intermittently in the past, however, tells me I should replace any remaining H264 videos, as they certainly use the High profile.

Thanks!

Alan

[Sam Kusnetz]

Alan Harvey wrote:
> Useful info here. I normally produce videos in PP, where I can export
> it any way I want; rarely have I just taken a video from elsewhere and
> popped it into QLab without messing with it, and re-exporting it, in
> PP. Live and learn!

I'm glad I could help! What's PP? I generally create in Final Cut and
export using Compressor.

> Also interesting to learn that QLab has problems with H264 High
> Profile; PP (at least PP6) doesn't offer Extended, and Main and
> Baseline at least sound inferior to High, but not if they work and
> High doesn't!

The names are pretty frustrating to me. It's true that high profile is
"better" than main, extended, and baseline, but the purpose of each
profile is pretty enlightening. High profile is intended for Blu-Ray
encoding, which uses purpose-built hardware and assumes that you're not
doing anything more exciting than pausing, fast forwarding, and
rewinding (ever spent much time fast forwarding and rewinding a Blu-Ray?
It's choppier than YouTube!). Main profile is designed for SD television
broadcast video, extended profile is intended for online streaming
video, and baseline profile is intended for "low-cost applications that
require additional data loss robustness" whatever that means.

> The (possibly) problematic mp4 file does use H264, but QT doesn't tell
> me which profile. The fact that this issue has happened intermittently
> in the past, however, tells me I should replace any remaining H264
> videos, as they certainly use the High profile.

A bit of the ol' re-encoding should do you there.

Best

[micpool]

On Tuesday, January 27, 2015 at 12:05:13 AM UTC, sam kusnetz wrote:

 What's PP?

 Adobe Premiere Pro?

[Sam Kusnetz]

micpool wrote:
> Adobe Premiere Pro?
Ha!

I use Premiere all the time, but I guess I never looked closely enough
to realize there's a "Pro" right there in the name.

Every day's a school day.

[micpool]

Used to be abbreviated as PP, but I think now the official Abbreviation is Pr CS6 to match the periodic table (Breaking Bad) logotype.

[Mike Post]

I wholly believe Sam’s assessment is as usual quite accurate, but we seem to be chasing around in codec inspired circles. When I started this party, PhotoJPEG was the weapon of choice. Animation was necessary for alpha channel work. Then it became h.264 because some video cards were supporting that and took load off the processor. Now Animation is gone in favor of ProRes4444 and 422LT is better than h.264. My head keeps spinning…

One question: are we still in a world where you need to choose a codec based as much on the capabilities of the box as the quality of the video or did AVFoundation change that? Would 422 work better on Alan’s mini than h.264? I’m headed into 2 shows with moderately heavy video on Minis and had decided to go with h.264 based on previous experiences. How do I know if that’s the right choice for these minis? Anyone got a chart for this kind of thing?

Side note: I’m insisting on V3 for these shows.

Mike Post
mdp...@mac.com
http://mdpostdesign.com
(601) 307-8657

[Andy Lang]

On Mon, Jan 26, 2015 at 6:12 PM, Alan Harvey
<aha...@georgiasouthern.edu> wrote:
> I've seen the occasional mention of this "hold last frame," but can't quite
> figure it out, and don't see anywhere in the Cue List workspace where it is
> mentioned or set. It's not something I've set for any video; is it a QLab
> parameter, or an attribute that can vary with incoming videos?

Sorry, Alan, that'll teach me to paraphrase without looking to see
what we actually call it in the program. What you're actually looking
for is "Hold at end", which can be found on the right side of the
"Time & Loops" tab of any video cue that's not a still image.

-Andy

[micpool]

There is one huge problem with h264 codecs. It is actually possible to have an h264 video that outperforms ProRes both in terms of image quality and playback performance. It is also just as possible to have an h264 video that will hardly play in Qlab at all. Trying to work out what compression settings will give you the former while avoiding the latter is near impossible. Therefore it is best avoided.

I repeat my earlier advice:

Life is too short to be constantly testing codecs

In Qlab2 h264 doesn't work very well

In Qlab 3 h264 can produce good results but it's quite easy to produce a h264 file that doesn't play well depending on your choice of compression options

A good aim is to get any recent mac to play the maximum number of simultaneous video files possible with good picture quality on projector output.

My suggestion for a one size fits all rule for which codecs to use is this:

The first codec to try when encoding files for Qlab is ProRes 422 (proxy) which gives good video quality on projector output under most circumstances.

If you require higher picture quality then use ProRes 422 (LT) but be aware this may halve  the number of simultaneous videos Qlab can handle.

If you need transparency use ProRes 444 but be aware the data rate for this codec is extremely high.

Here is a table I made ear lie  of data rates for the 5 flavours of ProRes Codecs and h264 rendered in After Effects at 50% Quality.r:

16:9 25fps Video Files Data Rates MB/s
	ProRes444	ProRes422(HQ)	ProRes422	ProRes422(LT)	ProRes422(proxy)	h264 50% qual
1080p	36	24	15	10	4.5	5
720p	18	12	7.5	5	2	2.5
480p	12	8	5	3.5	1.5	1

In experiments I have found a rough rule of thumb is to determine what your Qlab system can handle by seeing how many simultaneous 1080p individual videos you can run comfortably, and calculate the total data rate. You can then use the table to estimate how many files in codecs of greater or lesser data rates you might be able to run

e.g on my MacBook Pro system I can run 4 ProRes 422 (LT) 1080p videos totally reliably. So in theory I could run 8 720p videos in (LT)  or 16 720p videos in proxy codec.

A low spec Mac Mini might only manage a quarter of this performance. A Mac Pro with SSDs should  manage considerably more.

I always find when starting a project that a day testing using the likely combination of video dimensions, edge blends, screen geometries, video effects. alpha channel, and triple heads/Datapaths  etc. to determine what can reasonably be achieved with the hardware budget available saves hours later. I think it is also true to say that you need to leave a reasonable safety margin to get a show that will perform consistently without glitches night after night. It is very easy to overestimate the video performance you think you should achieve on any system!

In Activity monitor there is a CPU meter that will float above other applications called unsurprisingly, floating CPU meter. CPU History is also useful.  I would NOT recommend running Activity Monitor during a show but for testing it can be useful.

You can even have a script that launches Activity monitor on a hotkey

 It's also worth noting that a Quad core Mac has a CPU usage of up to 800 percent so CPU figures that exceed 100 percent are nothing to worry about.

Further reading: There are discussions of these issues on the following threads.

Qlab 3 ProRes vs h264 codecs. 

https://groups.google.com/forum/#!searchin/qlab/422$20proxy/qlab/RgECFNhIuJk/aXKahffTpZQJ

Qlab3 Video Codec Data rates and quality comparisons

https://groups.google.com/forum/#!searchin/qlab/Qlab3$20Video$20Codec$20Data$20rates$20and$20quality$20comparisons$20/qlab/aEF6jIoFBbg/kKmEgJHDbHYJ

Mic

[Mike Post]

Thanks Mic.  I do remember your discussion about not spending too much time chasing codecs.  I also took to heart luckydave’s comment that there’s no one good answer - you just have to try things and see what works best.

Didn’t spot 422 Proxy before - I’ll have to play with it.

Mike Post

mdp...@mac.com

http://mdpostdesign.com

(601) 307-8657