Frame Capture From Video

[Germaine Greenweig]

Myapplication performs several rendering operations on the first frame (I am using Metal, although I think the same applies to GLES). For example, it renders to targets that are used in subsequent frames, but not updated after that. I am trying to debug some of draw calls from these rendering operations, and I would like to use the 'GPU Capture Frame' functionality to do so. I have used it in the past for on-demand GPU frame debugging, and it is very useful.

Unfortunately, I can't seem to find a way to capture the first frame. For example, this option is unavailable when broken in the debugger (setting a breakpoint before the first frame). The Xcode behaviors also don't seem to allow for capturing the frame once debugging starts. There also doesn't appear to even be an API for performing GPU captures, in Metal APIs or the CAMetalLayer.

To tell Xcode to begin capturing a frame, add a breakpoint in Xcode at a line in your code somewhere before the point at which you want to start capturing a frame. Right-click the breakpoint, select Edit Breakpoint... from the pop-up menu, and add a Capture GPU Frame action to the breakpoint:

To indicate the start of the frame to capture, before the first occurrence of MTLCommandBuffer presentDrawable:, you can use the MTLCommandQueue insertDebugCaptureBoundary method. For example, you could invoke this method as soon as you instantiate the MTLCommandQueue, to immediately begin capturing everything submitted to the queue. Make sure the breakpoint in item 1 will be triggered before the point this code is invoked.

To indicate the end of the captured frame, you can either rely on the first normal occurrence of MTLCommandBuffer presentDrawable:, or you can add a second invocation of MTLCommandQueue insertDebugCaptureBoundary.

Finally, the MTLCommandQueue insertDebugCaptureBoundary method does not actually cause the frame to be captured. It just marks a boundary point, so you can leave it in your code for future debugging use. Wrap it in a DEBUG compilation conditional if you want it gone from production code.

Help! I always use this feature to capture a shot from some video footage from my DJI Phantom. It saves me time of having to snap lots of still pictures. When I used to use the "capture frame" drop down menu from the video playback bar in the library function, the frame I would capture would appear right next to the video in the line of photos at the bottom of the window. Now it shows up as a stack, and even though I have tried all the options under stacking when I right click on the selected frame, I only get a change in the way the stack icon shows up. I now don't see the captured file anywhere. This has only just recently been a problem so I am assuming an update changed something in my settings, or this is a bug. On a scale of 1-10 of my knowledge of this program, I am between a 2-3. So, if I am missing something simple, please keep that in mind. Thanks!

It may be that you have a filter set that is preventing the capture frame from showing up in the filmstrip at the bottom. Make sure that Library > Enable Filters is unchecked and then try capturing the frame again.

Next, it may be that stacks are collapsed by default. Do Photo > Stacking > Expand All Stacks. Now when you capture a frame, it should appear in a stack at the bottom, but with the stack already expanded.

Finally, if you are currently viewing a collection or smart collection that contains the video, that is, if the collection is selected as a source in the Collections panel in the left column of Library, then when you capture a frame, it won't show up at all, because LR doesn't automatically add the frame photo to the collection.

I have the same issue, it's not a that the video is in a collection or that Filters is unchecked. When you import it puts your files in a Import "bucket" that acts like a collection. If you right click on any image and select "go to folder in library" you will see the image you captured from video as expected.

Thanks, that's an one more thing to check. When you import, by default the current source is changed to Previous Import in the Catalog panel on the left-hand side. As you observed, with respect to stacking, Previous Import acts like a collection, and stacks are specific to the collection or folder in which the stack is created. Capture Frame creates a stack in the folder containing the video, so you can only see that stack when you set the current source to the folder or All Photographs. (One more wart caused by LR's weird stacking design.)

The only way I could get the captured frame to appear was by Creating a Collection with the video inside the collection, capturing the frame I wanted while I was in the Collection, then going to Library>Grid View. Then and only then did the captured frame appear.

My OS X app is successfully rendering content to the view, and that content is visible and dynamic in the app window. That must be distinctly understood, or nothing wonderful can come of this tale (with apologies to Mr. Dickens).

When I attempt to use Metal frame capture, things get messed up. The rendering commands are captured correctly, but all attachments (color, depth, etc), displayed by clicking on any draw command selection in the frame stack, are blank, and show no content at all. The captured scene on the running app also vanishes. In addition, inspecting vertex attribute buffers shows no content either. But index and uniform buffers are populated.

To reiterate...the app is rendering content correctly, the frame is captured accurately in Xcode, I can successfully inspect all the commands, render state, and bound objects in the frame capture, but all framebuffer attachments and vertex attribute buffers are empty.

This is not my first visit to the frame capture rodeo, and many very similar apps are capturing correctly. I've been digging for hours, but can't seem to find any difference between apps that are captured correctly and this app, where the framebuffers and vertex attributes are not captured.

The issue was that the app that was causing the trouble with Xcode Metal frame capture is using Private memory storage for vertex attribute and index buffers. Changing vertex buffers to Managed memory fixed the issues with frame capture. It was not immediately obvious that this should be the cause, because the content of the vertex index buffer does appear in the frame capture, even though the index buffer is using Private storage.

The second option is the preferred choice. Changing memory storage modes within the app, just to be able to use frame capture, is inconvenient at best, and somewhat risky at worst, as it relies on the developer remembering to toggle back, or suffer possible unexpected performance degredation. The Metal Debug classes should really handle this under the covers automatically.

I know it's possible to take a screenshot of video, but I'm not sure if that ends up being a lower resolution image. Also, it just feels clumsy to use a screenshot because HD the time I end up with UI controls over top the image.

This can be done with an app called Frame Grabber. It captures still frames from a video, maintaining both original quality up to 4K and metadata (datetime, GPS...). It used to be free, but now costs $20 per year.

Depending on the video resolution you may or may not end up with a lower resolution screenshot. If your video resolution is higher than the screenshots' resolution, you end up with a lower resolution image in the end. Read more about video resolution on iPhones and screenshot sizes.

There are many things to like about QuickTime X, and many improvements were included over QuickTime Pro 7... but there were also a ton of features removed (like being able to set advanced export options, save a movie as images, and do some other more advanced edits/exports.

However, I'm glad I finally figured out how I can grab one frame from a movie in QuickTime Player X. The problem I was having is this: If you pause the video and use the left/right arrow keys to move the playhead exactly to the frame you want, the player controls are still showing over the video (in addition to the video title bar/window chrome.

There still aren't many great options for exporting in QTX, but you can still download and install QuickTime Pro 7 (it's even included on the OS X Snow Leopard Install DVD!) if you need those advanced features.

Jeff, I so want to meet you at some point. Your help with introducing me to Carson and Joel has been invaluable for my company, and then I Google a borderline random Mac question and you're the solution. Bravo good sir!

The frame capture error occurs on the frame skipping command right after the sensor reset, so before taking an image and saving it. Even with a 1 second delay before the sensor initialization sequence, the frame capture still times out at the same rate.

3a8082e126