I can strongly recommend using Claude or any other strong AI to understand your camera and optics. I had a similar problem with optics and I got a great result from it. The right distance for a given lens, the right angel of view and expected focus depth, wich depends a lot from light and aperture.
Finally I chose 6mm for down looking and 16mm for upwards. The angel of view is around 15x23 mm which is small but gives good precision and large enough to capture all packages and fiducials.
Furthermore, I would recommend using a monochrome camera with global shutter. This is by far the most precise option.