Syntheyes Ground Plane

4 views

Skip to first unread message

Lucy Ginsburg

unread,

Aug 3, 2024, 5:21:50 PM8/3/24

to tarerore

It seems like it would save resources if you were able to 3d camera track specific regions of your footage. Can this be done using masks? Or do you have to track the whole scene then eliminate track points after the fact?

The whole point of a 3D tracker is to determine an actual camera placment, so in order to determine sufficient parallax, the points have to be spread far and wide and there have to be quite a few of them. Therefore masking out entire areas doesn't even make sense - if you were to limit the tracking to a cropped region, the solution would be very likely extremely off since the solver doesn't have enough info to reconstruct things like lens curvature and relative movement of the tracked points based thereon. Masking and cropping before a track therefore an absolute no-no. Of course manually placing and eliminating autoimatically generated track points after the initial analysis is a different story, but if you need that, you have to look at tools like SynthEyes. The rest is mostly a moot point - most 3D track solvers don't care for the actual number of points. That's simply not how this stuff works. Forcibly reducing the number of tracked features would be more a matter of the human side of things, but not a way to optimize the tracker. Even AE's limited tracker can handle a few hundred points without issues and most commercial alternatives can track humungously large scenes with potentially millions of track points.

I did a project once where I used Mocha AE to corner pin track the left cheek of an actor, expanded the surface, applied the corner pin track to a solid, applied a grid to the solid, pre-composed just the grid layer, camera tracked the Grid, established an origin and ground plane, added a solid and a camera, used the parenting trick to move both the solid and the camera so the solid was at comp center, exported the camera and solid as a C4D file, added a 3D model I created in Blender of really badly burned face with the jaw bone exposed, then imported that into AE and gave my actor a truly disgusting left side of his face that was a completely believable 3D overlay on the face.

Let us know exactly what you are trying to achieve and there is probably a solution. If it is nothing more than surface replacement then the basic Camera Tracker is or Mocha AE, or even corner pin tracking is probably what you want to use. If it is more complex then the existing tools can probably be used in a more creative way to pull off the shot you are trying to create.

So, i work with drone footage of golf courses. Placing text on the ground plane is often essential but when all of the ground is the same color it is frequently overlooked by the 3d camera tracker. Ill get track points on the trees and ther signifigant objects but very few or no track points on the ground.

Try this. Duplicate your footage, add some significant color correction to the duplicate to bring out as much contrast and detail as you can and don't worry about how it looks (I recently applied Colorama to a scene to get a decent track and it was all bright green, orange, and blue, but at least I had some detail to track) Pre-compose the color corrected footage, Camera track the Pre-comp, Make sure you establish an origin and ground plane then add a camera and Solid using the same points, add a Grid to the solid to make sure that the track is good, then start selecting track points for the areas you need to add graphics to and try and keep your points as far apart as you can to make sure the surface track is accurate. I usually put solids everywhere I want to add something, add a grid and make sure the track is good, then I add my graphics or replacement layers, make them 3D and parent them to the reference solids while holding down the shift key. This snapps the replacement layer to the same position and orientation as the reference solid. I then just turn off the solids or delete them.

If you can't get camera tracker to work properly you can still use Mocha AE, Corner Pin track or even Motion Stabilize just about any area of the golf course and then insert your graphics and put the motion back in the shot. I've done that kind of trick many many times on shots that just did not lend themselves to camera tracking.

I tracked a video succesfully in Voodoo and imported the 3D point cloud and virtual camera to Blender. I identified four points that were tracked to the ground floor in the video and made a plane out of them in Blender.

The question is, what should I do in Blender in order to make this floor aligned with the x-axis (in other words so that I see it straight from above in the Top view)? I realize that I would have to rotate pretty much everything else too to have the realtionships of the 3D points and the camera stay correct.

I have cut my video in three pieces since the analyze of the footage for the 3D camera-tracker failed. I followed this tut: =ZCHMyUqmIYI and created a null instead of a text-layer. I have attached my animation on the middle clips null and positioned the animation in the frame. All good. But when I attach a duplicate of the animation to attach that to the other null for the first clip - the positions are all wrong.

I just can't figure out how to make the transition to match! I think I'm making it in the wrong order cause I saw the position varies if I make the layer 3D before or after I attach it to a null. Same goes with the null.

Long and short answer: This won't work with AE's 3D tracker. Every solve is arbitrary and uncalibrated. tracks from different segements will never line up. If that's required, you have to move on to greener pastures and use tools like SynthEyes or mochaPro. Otherwise you have to limit yourself to stuff that can be seen in each segment and kinda create an illusion of a contiguous 3D environment by scaling and aligning the Nulls, text layers and whatever per segment based on the associated camera for the clip, but it will be nigh on impossible to actually get a consistent single camera spanning the entire comp.

The way to cut up a clip that will not camera track so you can use different parts of the video to track different surfaces at different times is to cut up the video, run a camera track on each part, make sure you set the ground plane and origin and add a reference solid and a camera there to verify the track is accurate, add the object that you want to track to that section of the video, then run the same process again on a different portion of the video. This is easiest to do if you pre-compose the cuts, turn the footage into a guide layer, make the composite, then use the pre-comps in the main comp to put the whole scene together. I carefully explained the workflow in this post: motion tracking help

To get a perfect match you'll have to overlap of the same graphic in different parts of the shot I would overlap the copy of the footage by 1 frame, use the pre-comp workflow I described in the post, temporarily set the blend mode to difference on the top nested comp, set up two comp views and two timeline views, then carefully match the position and rotation of the second graphic by manipulating the values carefully in the second shot pre-comp. It can be done. I've done it several times. When the difference mode tells you that you have matched the position, set the blend mode back to normal and trim the overlap. You'll never see the transition between cameras.

There is nothing wrong with the idea, the tutorial you followed was presented with an amateur that stumbled on a solution that kind of worked for his specific shot. Make sure you take advice from folks that know what they are talking about.

SynthEyes offers complete control over the tracking process for challenging shots, including an efficient workflow for supervised trackers, combined automated/supervised tracking, offset tracking, incremental solving, rolling-shutter compensation, a hard and soft path locking system, distance constraints for low-perspective shots, and cross-camera constraints for stereo. A solver phase system lets you set up complex solving strategies with a visual node-based approach (not in Intro version). You can set up a coordinate system with tracker constraints, camera constraints, an automated ground-plane-finding tool, by aligning to a mesh, a line-based single-frame alignment system, manually, or with some cool phase techniques. The ViewShift system allows you to do object removals, combine split takes, generate animated texture maps, and more.

Accordingly, the legacy Fusion exporter now supports 3D planar trackers; primitive, imported, or tracker-built meshes; imported or extracted textures; multiple cameras; and lens distortion via image maps.

The new lens distortion feature should make it possible to reproduce the distortion patterns of any real-world lens without its properties having been coded explicitly in the software or a custom plugin.

I am using the camera tracker from the foundry, but I am unable to find a solid ground plane. The average RMS error seems to be around 2.0. I can get it a bit lower, to 1.7, by removing some bad tracks. After that, when I create a solid from some points that seem to lie on the ground (tried different sets of points), the solid is unstable during the shot. My goal is just to place a 3d object on the grass.