Adding dual camera support to pandaboard
stereo
I want to add dual camera support on pandaboard for 3d video
processing.
Options i see with pandaboard
1) Use the two usb ports to connect to two usb cameras.
I see no issues with this on support in terms of hardware and
software.
But I feel it might not be the optimal method as it is not using ISS
or to make use of ISS I could waste some bandwidth, which could later
become bottleneck.
2) Use the camera board connector J17.
Understand that Pandaboard has already support (interms of hardware
and software-drivers) for single camera interface using ISS
hardware board: http://www.omappedia.org/wiki/PandaBoard_Camera_Support
software -driver: http://pandaboard.org/content/omap4-v4l2-camera
So ideally want to use the above resources and extend to stereo camera
support, so the development time can be reduced.
Few questions in achieving the objective.
1) Are CSI2-A(primary) and CSI2-B(secondary) port pins (control and
data pins) are exposed in the J17 header?
(I tried to compare the system reference manual of pandaboard and
omap4 technical reference manual and could not arrive at 1 to 1
relation of pins)
2) If only one camera port is exposed(CSI2-A), is it possible to
achieve the simultaneous real time video capture objective
(multiplexed)?
2) Is it possible to use the existing camera adapterboards for the
above objectives?
3) With respect to software, I beileve its the driver which need to be
adapted to achieve simulatneous video capture? Is this feasible?
Would also like to know any other optimal way to achive the above
objective or any other readymade solutions ?
Thanks in advance
Aguirre, Sergio
On Fri, Aug 26, 2011 at 7:23 AM, stereo <linux...@gmail.com> wrote:
> Hi all,
>
> I want to add dual camera support on pandaboard for 3d video
> processing.
>
> Options i see with pandaboard
>
>
> 1) Use the two usb ports to connect to two usb cameras.
> I see no issues with this on support in terms of hardware and
> software.
> But I feel it might not be the optimal method as it is not using ISS
> or to make use of ISS I could waste some bandwidth, which could later
> become bottleneck.
May not be the optimal, but might be your only choice for PandaBoard case.
See below for the reason.
>
> 2) Use the camera board connector J17.
> Understand that Pandaboard has already support (interms of hardware
> and software-drivers) for single camera interface using ISS
> hardware board: http://www.omappedia.org/wiki/PandaBoard_Camera_Support
> software -driver: http://pandaboard.org/content/omap4-v4l2-camera
>
> So ideally want to use the above resources and extend to stereo camera
> support, so the development time can be reduced.
>
> Few questions in achieving the objective.
>
> 1) Are CSI2-A(primary) and CSI2-B(secondary) port pins (control and
> data pins) are exposed in the J17 header?
If you look at this schematic:
http://pandaboard.org/sites/default/files/board_reference/A1/750-2152-002-sch_revd.pdf
Page 13
You'll see that only CSI21 (a.k.a. CSI2-A) is exposed.
> (I tried to compare the system reference manual of pandaboard and
> omap4 technical reference manual and could not arrive at 1 to 1
> relation of pins)
> 2) If only one camera port is exposed(CSI2-A), is it possible to
> achieve the simultaneous real time video capture objective
> (multiplexed)?
Hardly. You can probably achieve the multiplexing, but the problem
is that, you need 2 images to be captured synchronously, otherwise
you'll have 2 images with a big time offset (33ms for 30 fps).
> 2) Is it possible to use the existing camera adapterboards for the
> above objectives?
Sadly, no. This is a PandaBoard side limitation. If you had your own
custom OMAP4 Board, then yeah, you could have access to CSI2-B.
If you have some serious money to spend (~$2K USD), then you can
get a Blaze board, which already has a 3 camera sensor setup
(Two 5MPix sensors front facing, and One 12 MPix back facing).
Here: http://www.svtronics.com/market_omap
> 3) With respect to software, I beileve its the driver which need to be
> adapted to achieve simulatneous video capture? Is this feasible?
It should be feasible, it's just a matter of coming up with your design for S3D,
and holding a good discussion in linux-media mailing list. I'm pretty
sure people will be
happy to hear about a good proposal.
But basically, you'll need to have such a design that allows you to take care of
capturing sync between 2 sensors, and some major algorithmical work to achieve a
good 3D capture.
Now, this is valid for both having a USB Webcam setup, and a
OMAP4 ISS driver w/CSI2 sensors setup, so its really more about how to
define a good API to couple 2 streams as a stereoscopic capture.
How? I don't know. :) But I encourage you to think on a good design proposal. ;)
>
> Would also like to know any other optimal way to achive the above
> objective or any other readymade solutions ?
Not sure to be honest.
Regards,
Sergio
>
> Thanks in advance
>
harikumar gs
Aguirre, Sergio
On Mon, Aug 29, 2011 at 10:03 AM, harikumar gs <linux...@gmail.com> wrote:
> Hi Sergio,
> Thanks for you reply.
> I have few more queries which I would like to seek answers
> 1) When you say that "usb may not be the optimal", can I confirm that you
> are referring to band width issues arising due to memcopy to allow video
> processing by IVA-HD. If not what could be the other issues?
Well, first of all, IVA-HD encoders have specific requirements, like
having the image in NV12 format with a Page aligned stride (4KB offset
between every start of line), and this has to be in TILER DMM space as
far as I know.
So, You basically will receive either UYVY or MJPEG from the USB
Camera, and you'll need to transcode that, most likely in SW, so that
the right format is sent as an input to the encoders.
So, all that should be definitely an impact to the possible encoding
framerate, so you'll
have to consider that also, besides the memcpys involved.
> 2) We are ready to consider Blaze Development platform So have few
> clarifications discussed below to finalize Blaze.
> Without having low level access to IVA-HD (DSP+ hardware accelerators) real
> time 3d vision processing could become impossible. So does the Blaze
> development platform provides the required SDK(document and sample programs)
> for low level access to IVA-HD (DSP+hardware accelerators) for developing
> customized 3d vision processing. Please let me know any way of getting
> access to IVA-HD resources, by signing NDA etc. (Aware of TI providing OMX
> component to get access to IVA-HD at decoder/encoder level, which is not
> what is required in my case)
> 3) Does the Blaze development board have 3d driver support? (I have the
> desire to develop 3d driver, but not sure whether I am well equiped to do
> so in terms of experience. Hope I get the time and oppurtunity to work on
> this.)
Internally in TI, there are closed-source reference source codes to
have Stereoscopic 3D
Camera/Video. Of course this requires having contact with your TI
representative, and
definitely consulting with them how much access you could get to this.
Now, to be honest, as this has some TI Private IP, you'll probably
only get support from the
dedicated experts for doing the bring-up for you. But for that I'm not
100% sure, as it is not
my formal area of work.
However, if you want to keep things open source, and you want to be
able to follow this from a
video4linux perspective, you'll have to think on your own design from
scratch, and quite frankly
you'll be presenting a nice alternative, but you'll be practically
competing against TI SW IPs
pretty much :), so don't expect much support from the SW algorithms
point of view.
Other than this, I'm unfortunately incapable of sharing more details
about the S3D internals, hope
you'll understand.
Regards,
Sergio
Vladimir Pantelic
> Hi Hari,
>
> So, You basically will receive either UYVY or MJPEG from the USB
> Camera, and you'll need to transcode that, most likely in SW, so that
> the right format is sent as an input to the encoders.
>
> So, all that should be definitely an impact to the possible encoding
> framerate, so you'll
> have to consider that also, besides the memcpys involved.
converting he *current* frame from UYVY to NV12 can be done while the
encoder encodes the *previous* frame, the impact on encoding frame rate
should be negligible if done properly. And converting UYVY to NV12 can
be done very efficiently using neon...
stereo
Thanks to Vladimir for suggesting solution with Pandaboard.
Now I see that SW algorithm that you were talking about is all about
how to achieve good synchronization between the frames of the two
cameras.
ie. triggering the capture of video data exactly at the same instant
of time. Correct me if I got it wrong?
Have a clarification in the following
1) Will the open access documents/example codes currently provided by
TI, sufficient to start writing the stereo capture driver?
2) Can the same algorithm apply for both stereo capture using two usb
camera option and two CSI interface option?
Thanks and Regards,
Hari