CSI driver problems in 16bit mode + some successful debug

[Ivan Kozic]

Hi all,

I need help from someone who knows more about CSI driver. I want to use 16bit YUV422 capture mode for CSI0. However, I have found that this mode is not supported in kernel 3.4, so I went on to hack the kernel.
My system is Allwinner A20 on Olinuxino Micro board.
First problem is that 16-bit mode is never applied, even if csi_if is set to '1' in the fex file and define d8..d15. The problem is in sun4i_csi_core.h:

/*
 * input data format
 */
typedef enum tag_CSI_INPUT_FMT
{
    CSI_RAW=0,     /* raw stream  */
    CSI_BAYER,     /* byer rgb242 */
    CSI_CCIR656,   /* ccir656     */
    CSI_YUV422,    /* yuv422      */
}__csi_input_fmt_t;

So, in this enum, CSI_YUV422 is always set to 3, and as so written to the CSI0_CFG register. The correct mode for 16bit is 4, so the simplest way is to change the enum like:

/*
 * input data format
 */
typedef enum tag_CSI_INPUT_FMT
{
    CSI_RAW=0,     /* raw stream  */
    CSI_BAYER,     /* byer rgb242 */
    CSI_CCIR656,   /* ccir656     */
    CSI_YUV422=4,    /* yuv422      */
//    CSI_YUV422,    /* yuv422      */
}__csi_input_fmt_t;

However, surprise, surprise, this doesn't work (no capture at all). The second problem is that YUV422 16bit is supposed to be using embedded sync codes, unlike YUV422 8bit mode, which is not and much like bt.656 (CCIR656) mode, which by the way is not supported at all in the driver (just a bunch of todos). So, I have enabled embedded sync codes for my sensor system (sensor + fpga, so it's quite flexible), but there was a third problem - unlike 8bit mode, 16bit mode is latched on the opposite edge, so you need to define CSI_FALLING as clock polarity for whichever sensor driver you're using (or simply invert pixel clock), otherwise CSI will not capture anything (as the sync codes are messed up when latched on the wrong edge).

However, even though now I am successfully capturing the signal coming from the camera, I am only capturing the lower 8 bits - I can't get the upper 8 bits no matter what I do. I am currently suspecting DMA (seems likely) and/or GPIO config for 16bits (this also seems strange - I couldn't find any references to csi_d* vars from fex file in the entire kernel), but other than that I am running out of ideas quite fast. Debug up to this point was already hard enough, as there is no clear documentation from Allwinner about this at all (I have searched through their entire line-up - starting from A10, up to A31).

If someone knows something, please share - if I had some more documentation, I would have probably already made the whole thing work, but this way I feel like I am more and more just trying out some things blindly than actually solving problems...
So please help if you know more. If I get further, I will post my progress back of course.

[Martin Collins]

On 2014-03-14 10:42, Ivan Kozic wrote:
>
> If someone knows something, please share - if I had some more
> documentation, I would have probably already made the whole thing work,

I don't know anything, but by chance I was looking here today:
http://dl.linux-sunxi.org/A10/
My impression was that there is an 8 bit channel and a 24 bit channel.
So perhaps it won't do 16 bits?

Martin

[jons...@gmail.com]

I haven't tried any of this...

Look like you would set the input format...

22:20 R/W 3 INPUT_FMT
Input data format
000: RAW stream
001: reserved
010: CCIR656(one channel）
011: YUV422
100: YUV444（{R, B, G} or {Pr, Pb, Y}）

When the input format is set YUV444
1100: field planar YUV 444
1101: field planar YUV 422 UV combined
1110: frame planar YUV 444
1111: frame planar YUV 422 UV combined

The the output format is always 24b, right?

You're going to have to read the user manual. I don't believe anyone
has played with this before. But it looks like the hardware supports
it.

Also - the CSI driver in the sunxi tree is quite old. First thing I'd
do is update it using the most recent Allwinner CSI driver we can
locate. You might even get lucky and the newer Allwinner drivers could
support 16b.

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Jon Smirl
jons...@gmail.com

[Ivan Kozic]

Hi Martin,

Thanks for the reply, but I'm actually using A20, which supports 16bit YUV422 capture on CSI0 port. A10's CSI unit is slightly different as it's older, although many register settings are similar - just for this reason I was going through documentation for other Allwinner cpus, but in fact I'm interested mainly for A20.

[Ivan Kozic]

Hi John - just saw your post. I think you're looking at a wrong document, as for A20, the register setting 100 for bits 22:20 is YUV422 16bit - it's written in the A20 User Manual on page 371. The problem is in fact this very vague information in the User Manual - it's almost useless. This is why I'm asking for help from someone who maybe knows the kernel a bit more - for instance, I'm not even sure which DMA is used for transfers from CSI to memory, because I have to figure it out from the driver.

I'm also not sure where to search for the driver or more help. This group seems most professional of all by far - I have tried to contact Allwinner, but to no avail so far. As for sunxi kernel and drivers, I only know of the Git branch, but noone there has touched CSI driver for 8 months. Although I did always look in the 3.4 branch, maybe I should check out mainline...

Anyway, I'll search for the actual solution myself and if I find it post back - If anyone can speed me up, I'll be most grateful.

[jons...@gmail.com]

I copied from the A10 manual.

Here's a more recent driver....

[Ivan Kozic]

Hi John,

Wow thanks - this looks quite cool - it seems that they've started working on YUV422 16bit and CCIR656 as well. May I ask where did you find the driver?
One crucial file is missing - sun4i_csi_core.h or sunxi_csi_core.h I guess, as it's not for A10 anymore.

[jons...@gmail.com]

On Sat, Mar 15, 2014 at 11:29 AM, Ivan Kozic <jimm...@gmail.com> wrote:
> Hi John,
>
> Wow thanks - this looks quite cool - it seems that they've started working
> on YUV422 16bit and CCIR656 as well. May I ask where did you find the
> driver?

It is from the current Allwinner Android tree. The CSI driver in sunxi
is about two years old.

sunxi kernel needs to be updated to use this driver.

[Ivan Kozic]

Hi John - could you give me a link to this Android tree? I seem to have difficulties finding it on GitHub...

--
You received this message because you are subscribed to a topic in the Google Groups "linux-sunxi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/linux-sunxi/vU5-3Pc3iOs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to linux-sunxi...@googlegroups.com.

[jons...@gmail.com]

On Sun, Mar 16, 2014 at 6:52 PM, Ivan Kozic <jimm...@gmail.com> wrote:
> Hi John - could you give me a link to this Android tree? I seem to have
> difficulties finding it on GitHub...

It is in one of the sunxi download directories. I forgot which one I
got it from. Maybe the Olimex directory.

It is 4.2GB file.

[Ivan Kozic]

Hi John,

I've only found it here:

https://github.com/qubir/PhoenixA20_linux_sourcecode/tree/master/drivers/media/video/

but this is not even forked from anywhere. Also it seems to be a normal linux source - although, as far as I can see, both android and linux kernels are integrated into this source.
Although I have no experience with Android at all, so I might be wrong.

I will either try to build this one, or just update my own CSI driver with the files from here, although that might not be a good idea, but I'll try either way.

If you remember where you've got the files from at some point, please post back - I don't like forking unknown kernel sources, and this qubir one seems half-legit somehow...

[jons...@gmail.com]

On Mon, Mar 17, 2014 at 6:36 AM, Ivan Kozic <jimm...@gmail.com> wrote:
> Hi John,
>

> I've only found it here:
>
> https://github.com/qubir/PhoenixA20_linux_sourcecode/tree/master/drivers/media/video/

It is somewhere on the download server:
http://dl.linux-sunxi.org/

[Ivan Kozic]

It might be somewhere in the Android SDK on this download server (it seems that only SDK has source files - everything else is either documentation or binaries).
Either way, I'm very surprised that people from linux-sunxi git repo are ignoring this, as official git repo for sunxi still has the old driver (which is pretty much unusable).

Also ov5640 driver is much better in the qubir link that I've sent you - not sure if it can be used for our stuff, as it seems that V4L2 integration is a bit different. The same goes for CSI unfortunately, so I'll probably have to use heavily modified kernel from git.

By the way - on the download server that you've sent there is a nice document about A10's CSI unit - I'm just preparing to read it.

Of course, if I find anything, I'll post back.

[Ivan Kozic]

Very happy to say that I've got it :)

The solution is actually very simple - DMA seems to work just fine, the problem is with GPIOs - even though we initialize fex file with csi_d0..csi_d15, it seems that only settings for d0..d7 are applied.
For quick check, I've modified csi_probe function found in sun4i_drv_csi.c like this:

/*pin resource*/
    dev->csi_pin_hd = gpio_request_ex("csi0_para",NULL);
    if (dev->csi_pin_hd==-1) {
        csi_err("csi0 pin request error!\n");
        ret = -ENXIO;
        goto err_irq;
    }
   
    // IKfix - override pin muxing just in case
    writel(0x33333333, 0xf1c20800+0x90);    // this is really just in case, as PE PIO group works
    writel(0x00003333, 0xf1c20800+0x94);    // this is really just in case, as PE PIO group works
    writel(0x55553333, 0xf1c20800+0xd8);    // this is for PG PIO group - configure PG gpios as CSI0
    writel(0x00005555, 0xf1c20800+0xdc);    // this is for PG PIO group - configure PG gpios as CSI0

Basically, just after the gpio_request_ex() has finished its work (by the way, this function should have already did this, but for some reason it didn't), I added a few lines of just-in-case configuration. First two are for lower 8 bits (d0..d7) - these are really just in case, as I've previously verified that it works. Second two are for upper 8 bits (d8..d15), and this is the culprit - gpio_request_ex should have already did this, as it's in the fex file and a correct number of gpios is reported (for my config - 22).

One problem solved.

I would have figured out gpios last week, but something in the kernel is not letting me access PIO registers via /dev/mem from userland - I always get seg faults when trying to access PIO base (0xf1c20800) and all the subsequent registers. As I thought DMA is the issue, I gave up on finding the culprit for the seg fault. Maybe some protection mechanism in action or similar.

[Ivan Kozic]

One more update - it seems that the error comes from me :) - I've used bad multiplex options for d8..d15 pins (should be 5, but copy-paste from CSI1 made it 3 - standard copy-paste mistake...).
Anyway, all the other fixes that I've posted should be implemented, as it will not work without them. Only the last one is my fault.

[jons...@gmail.com]

Yes, the kernel source is inside the Android SDK downloads. I can send
you files if you need them so that you don't have to install the 4.2GB
thing.

I was able to get my camera working. I then got stuck on the h.264
encoder. The encoder works but it is not compressing enough. No matter
what parameters I give it the output stream is still 5-10Mb/s. I need
a stream less than 1Mb/s.

[Ivan Kozic]

Cool, I was under the impression that VPU is quite messy to start on these Allwinners - I'll probably have a follow-up project where I would use VPU as well, so it's very good that someone's got it working.

Just wanted to say that it would be good to check the register settings for VPU, but in fact I cannot find any registers for VPU in the User Manual...

To unsubscribe from this group and all of its topics, send an email to linux-sunxi...@googlegroups.com.

[ditma...@gmail.com]

have you had a look at the A10 VPU registers ?
http://linux-sunxi.org/A10_Register_guide#Video_Engine_Registers

[Ivan Kozic]

This sunxi HW/SW/Docu combo is a complete mess. There's one more thing specific to YUV422 16bit capture - memory alignment is a bit different than when using 8bit capture. In fact, up to now, I've been using planar capture, which in turn means that Y, U and V components are stored into separate buffers (planes) => 3 addresses are generated, one for each buffer. So for example memory would look like:

0x10: Frame 1 Y plane
0x20: Frame 1 U plane
0x30: Frame 1 V plane
0x40: Frame 2 Y plane
...

However, for 16bit capture when using the same CSI setting, I get UV_combined subcase - Y is stored in one buffer/plane, while U/V is stored in the other (so one plane for both U and V) => 2 addresses are generated. Memory:

0x10: Frame 1 Y plane
0x20: Frame 1 U/V plane
0x30: Frame 2 Y plane
...

Admittedly, my fixes are quite dirty and some things might not really be optimal in my kernel (V4L MBUS codes are completely wrong for instance), but I still think that this alignment is one of the undocumented things by Allwinner - this alignment is set via CSI_CONF register - OUTPUT_FMT bits, but YUV422 16bit is not even mentioned here, so...

Anyway, the reason that I'm mentioning this is that display driver needs correct info about alignment to work properly - I'm still trying out stuff here, since there is no SW user manual on how to properly use display ioctls in case of UV combined mode, but I don't think it will be a big issue to start it up correctly. Will post back when I get it right.

> email to linux-sunxi+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
Jon Smirl
jons...@gmail.com

--
You received this message because you are subscribed to a topic in the Google Groups "linux-sunxi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/linux-sunxi/vU5-3Pc3iOs/unsubscribe.

To unsubscribe from this group and all of its topics, send an email to linux-sunxi+unsubscribe@googlegroups.com.

[Ivan Kozic]

Got it - instead of V4L2_PIX_FMT_YUV422P, CSI format needs to be set to V4L2_PIX_FMT_NV16. Also display needs to set as:

format: DISP_FORMAT_YUV422
mode: DISP_MOD_NON_MB_UV_COMBINED
seq: DISP_SEQ_UVUV

works like a charm :)

[zs.g...@gmail.com]

Hi,

How did you managed to make work the new driver from the Android source? Or you used the old one with the old CSI driver as well?
I need a driver for Samsung S5K4EC camera, and there is not even source for sunxi-linux :(

[Ivan Kozic]

Hi,

Easy - I didn't :) I used some code from the new OV5640 driver (AF mostly), but due to all the changes I did, I've decided to stick with the old driver. Same goes for CSI. In fact new CSI driver only has some kernel support for still image capture + somewhat better implementation for 16bit mode (but still not finished) - here I used my old code for 16bit mode, but still image capture from kernel was a bit unclear to me, so I just hacked V4L2 a bit and used V4L2 for still image capture.

By the way both the new and the old CSI/OV5640 drivers have their own issues - whichever you use, you'll need to do some code fixes. Regarding S5K4EC - I can see S5K6AA driver in /drivers/media/video/, but this is not implemented for CSI. However, it shouldn't be that hard to take this driver or OV5640 driver and rewrite it for S5K4EC (I did this for Aptina sensors for i.MX6) - it's basically just changing sensor settings and I2C addresses.

[zs.g...@gmail.com]

Hi Ivan,

I made some progress, but I post it on the relevant topic, could you please have a look:
https://groups.google.com/forum/#!topic/linux-sunxi/cjFOOLJSWpc
I got some issue, if it's been sorted out before then some help would be appreciated.
Thanks
Zsolt