Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
[PATCH v5 29/39] ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
580 views
Skip to first unread message
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
Enable i.MX v4l2 media staging driver. For video capture on i.MX, the
video multiplexer subdev is required. On the SabreAuto, the ADV7180
video decoder is required along with i2c-mux-gpio. The Sabrelite
and SabreSD require the OV5640 and the SabreLite requires PWM clocks
for the OV5640.
Increase max zoneorder to allow larger video buffer allocations.
Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
---
arch/arm/configs/imx_v6_v7_defconfig | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm/configs/imx_v6_v7_defconfig b/arch/arm/configs/imx_v6_v7_defconfig
index eaba3b1..5d8c94a 100644
--- a/arch/arm/configs/imx_v6_v7_defconfig
+++ b/arch/arm/configs/imx_v6_v7_defconfig
@@ -51,6 +51,7 @@ CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_AEABI=y
CONFIG_HIGHMEM=y
CONFIG_CMA=y
+CONFIG_FORCE_MAX_ZONEORDER=14
CONFIG_CMDLINE="noinitrd console=ttymxc0,115200"
CONFIG_KEXEC=y
CONFIG_CPU_FREQ=y
@@ -181,6 +182,7 @@ CONFIG_SERIAL_FSL_LPUART=y
CONFIG_SERIAL_FSL_LPUART_CONSOLE=y
# CONFIG_I2C_COMPAT is not set
CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_MUX=y
CONFIG_I2C_MUX_GPIO=y
# CONFIG_I2C_HELPER_AUTO is not set
CONFIG_I2C_ALGOPCF=m
@@ -221,14 +223,20 @@ CONFIG_REGULATOR_PFUZE100=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_RC_SUPPORT=y
+CONFIG_MEDIA_CONTROLLER=y
+CONFIG_VIDEO_V4L2_SUBDEV_API=y
CONFIG_RC_DEVICES=y
CONFIG_IR_GPIO_CIR=y
CONFIG_MEDIA_USB_SUPPORT=y
CONFIG_USB_VIDEO_CLASS=m
CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_MULTIPLEXER=y
CONFIG_SOC_CAMERA=y
CONFIG_V4L_MEM2MEM_DRIVERS=y
CONFIG_VIDEO_CODA=y
+# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set
+CONFIG_VIDEO_ADV7180=m
+CONFIG_VIDEO_OV5640=m
CONFIG_SOC_CAMERA_OV2640=y
CONFIG_IMX_IPUV3_CORE=y
CONFIG_DRM=y
@@ -338,6 +346,9 @@ CONFIG_FSL_EDMA=y
CONFIG_IMX_SDMA=y
CONFIG_MXS_DMA=y
CONFIG_STAGING=y
+CONFIG_STAGING_MEDIA=y
+CONFIG_VIDEO_IMX_MEDIA=y
+CONFIG_COMMON_CLK_PWM=y
CONFIG_IIO=y
CONFIG_VF610_ADC=y
CONFIG_MPL3115=y
--
2.7.4
video multiplexer subdev is required. On the SabreAuto, the ADV7180
video decoder is required along with i2c-mux-gpio. The Sabrelite
and SabreSD require the OV5640 and the SabreLite requires PWM clocks
for the OV5640.
Increase max zoneorder to allow larger video buffer allocations.
Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
---
arch/arm/configs/imx_v6_v7_defconfig | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/arch/arm/configs/imx_v6_v7_defconfig b/arch/arm/configs/imx_v6_v7_defconfig
index eaba3b1..5d8c94a 100644
--- a/arch/arm/configs/imx_v6_v7_defconfig
+++ b/arch/arm/configs/imx_v6_v7_defconfig
@@ -51,6 +51,7 @@ CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_AEABI=y
CONFIG_HIGHMEM=y
CONFIG_CMA=y
+CONFIG_FORCE_MAX_ZONEORDER=14
CONFIG_CMDLINE="noinitrd console=ttymxc0,115200"
CONFIG_KEXEC=y
CONFIG_CPU_FREQ=y
@@ -181,6 +182,7 @@ CONFIG_SERIAL_FSL_LPUART=y
CONFIG_SERIAL_FSL_LPUART_CONSOLE=y
# CONFIG_I2C_COMPAT is not set
CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_MUX=y
CONFIG_I2C_MUX_GPIO=y
# CONFIG_I2C_HELPER_AUTO is not set
CONFIG_I2C_ALGOPCF=m
@@ -221,14 +223,20 @@ CONFIG_REGULATOR_PFUZE100=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_RC_SUPPORT=y
+CONFIG_MEDIA_CONTROLLER=y
+CONFIG_VIDEO_V4L2_SUBDEV_API=y
CONFIG_RC_DEVICES=y
CONFIG_IR_GPIO_CIR=y
CONFIG_MEDIA_USB_SUPPORT=y
CONFIG_USB_VIDEO_CLASS=m
CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_MULTIPLEXER=y
CONFIG_SOC_CAMERA=y
CONFIG_V4L_MEM2MEM_DRIVERS=y
CONFIG_VIDEO_CODA=y
+# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set
+CONFIG_VIDEO_ADV7180=m
+CONFIG_VIDEO_OV5640=m
CONFIG_SOC_CAMERA_OV2640=y
CONFIG_IMX_IPUV3_CORE=y
CONFIG_DRM=y
@@ -338,6 +346,9 @@ CONFIG_FSL_EDMA=y
CONFIG_IMX_SDMA=y
CONFIG_MXS_DMA=y
CONFIG_STAGING=y
+CONFIG_STAGING_MEDIA=y
+CONFIG_VIDEO_IMX_MEDIA=y
+CONFIG_COMMON_CLK_PWM=y
CONFIG_IIO=y
CONFIG_VF610_ADC=y
CONFIG_MPL3115=y
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
From: Russell King <rmk+k...@armlinux.org.uk>
Add the bayer formats to imx-media's list of supported pixel and bus
formats.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
- added a bayer boolean to struct imx_media_pixfmt.
drivers/staging/media/imx/imx-media-utils.c | 68 +++++++++++++++++++++++++++++
drivers/staging/media/imx/imx-media.h | 1 +
2 files changed, 69 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index ee50b6a..24e3795 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -61,6 +61,74 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.cs = IPUV3_COLORSPACE_RGB,
.bpp = 32,
.ipufmt = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ .codes = {MEDIA_BUS_FMT_SBGGR8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ .codes = {MEDIA_BUS_FMT_SGBRG8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ .codes = {MEDIA_BUS_FMT_SGRBG8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ .codes = {MEDIA_BUS_FMT_SRGGB8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR16,
+ .codes = {
+ MEDIA_BUS_FMT_SBGGR10_1X10,
+ MEDIA_BUS_FMT_SBGGR12_1X12,
+ MEDIA_BUS_FMT_SBGGR14_1X14,
+ MEDIA_BUS_FMT_SBGGR16_1X16
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG16,
+ .codes = {
+ MEDIA_BUS_FMT_SGBRG10_1X10,
+ MEDIA_BUS_FMT_SGBRG12_1X12,
+ MEDIA_BUS_FMT_SGBRG14_1X14,
+ MEDIA_BUS_FMT_SGBRG16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG16,
+ .codes = {
+ MEDIA_BUS_FMT_SGRBG10_1X10,
+ MEDIA_BUS_FMT_SGRBG12_1X12,
+ MEDIA_BUS_FMT_SGRBG14_1X14,
+ MEDIA_BUS_FMT_SGRBG16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB16,
+ .codes = {
+ MEDIA_BUS_FMT_SRGGB10_1X10,
+ MEDIA_BUS_FMT_SRGGB12_1X12,
+ MEDIA_BUS_FMT_SRGGB14_1X14,
+ MEDIA_BUS_FMT_SRGGB16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
},
/*** non-mbus formats start here ***/
{
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 2dd48ea..9e865fa 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -91,6 +91,7 @@ struct imx_media_pixfmt {
int bpp; /* total bpp */
enum ipu_color_space cs;
bool planar; /* is a planar format */
+ bool bayer; /* is a raw bayer format */
bool ipufmt; /* is one of the IPU internal formats */
};
--
2.7.4
Add the bayer formats to imx-media's list of supported pixel and bus
formats.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
- added a bayer boolean to struct imx_media_pixfmt.
drivers/staging/media/imx/imx-media-utils.c | 68 +++++++++++++++++++++++++++++
drivers/staging/media/imx/imx-media.h | 1 +
2 files changed, 69 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index ee50b6a..24e3795 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -61,6 +61,74 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.cs = IPUV3_COLORSPACE_RGB,
.bpp = 32,
.ipufmt = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ .codes = {MEDIA_BUS_FMT_SBGGR8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ .codes = {MEDIA_BUS_FMT_SGBRG8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ .codes = {MEDIA_BUS_FMT_SGRBG8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ .codes = {MEDIA_BUS_FMT_SRGGB8_1X8},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 8,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR16,
+ .codes = {
+ MEDIA_BUS_FMT_SBGGR10_1X10,
+ MEDIA_BUS_FMT_SBGGR12_1X12,
+ MEDIA_BUS_FMT_SBGGR14_1X14,
+ MEDIA_BUS_FMT_SBGGR16_1X16
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG16,
+ .codes = {
+ MEDIA_BUS_FMT_SGBRG10_1X10,
+ MEDIA_BUS_FMT_SGBRG12_1X12,
+ MEDIA_BUS_FMT_SGBRG14_1X14,
+ MEDIA_BUS_FMT_SGBRG16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG16,
+ .codes = {
+ MEDIA_BUS_FMT_SGRBG10_1X10,
+ MEDIA_BUS_FMT_SGRBG12_1X12,
+ MEDIA_BUS_FMT_SGRBG14_1X14,
+ MEDIA_BUS_FMT_SGRBG16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB16,
+ .codes = {
+ MEDIA_BUS_FMT_SRGGB10_1X10,
+ MEDIA_BUS_FMT_SRGGB12_1X12,
+ MEDIA_BUS_FMT_SRGGB14_1X14,
+ MEDIA_BUS_FMT_SRGGB16_1X16,
+ },
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 16,
+ .bayer = true,
},
/*** non-mbus formats start here ***/
{
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 2dd48ea..9e865fa 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -91,6 +91,7 @@ struct imx_media_pixfmt {
int bpp; /* total bpp */
enum ipu_color_space cs;
bool planar; /* is a planar format */
+ bool bayer; /* is a raw bayer format */
bool ipufmt; /* is one of the IPU internal formats */
};
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
The cropping rectangle was being modified by the output pad's
set_fmt, which is the wrong pad to do this. The crop rectangle
should not be modified by the output pad set_fmt. It instead
should be reset to the full input frame when the input pad format
is set.
The output pad set_fmt should set width/height to the current
crop dimensions, or 1/2 the crop width/height to enable
downscaling.
So the other part of this patch is to enable downscaling if
the output pad dimension(s) are 1/2 the crop dimension(s) at
csi_setup() time.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
drivers/staging/media/imx/imx-media-csi.c | 35 ++++++++++++++++++++-----------
1 file changed, 23 insertions(+), 12 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index a7d04e4..577038e 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -536,6 +536,10 @@ static int csi_setup(struct csi_priv *priv)
ipu_csi_set_window(priv->csi, &priv->crop);
+ ipu_csi_set_downsize(priv->csi,
+ priv->crop.width == 2 * outfmt->width,
+ priv->crop.height == 2 * outfmt->height);
+
ipu_csi_init_interface(priv->csi, &sensor_mbus_cfg, &if_fmt);
ipu_csi_set_dest(priv->csi, priv->dest);
@@ -932,15 +936,15 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
switch (sdformat->pad) {
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
- crop.left = priv->crop.left;
- crop.top = priv->crop.top;
- crop.width = sdformat->format.width;
- crop.height = sdformat->format.height;
- ret = csi_try_crop(priv, &crop, sensor);
- if (ret)
- goto out;
- sdformat->format.width = crop.width;
- sdformat->format.height = crop.height;
+ if (sdformat->format.width < priv->crop.width * 3 / 4)
+ sdformat->format.width = priv->crop.width / 2;
+ else
+ sdformat->format.width = priv->crop.width;
+
+ if (sdformat->format.height < priv->crop.height * 3 / 4)
+ sdformat->format.height = priv->crop.height / 2;
+ else
+ sdformat->format.height = priv->crop.height;
if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
cc = imx_media_find_format(0, sdformat->format.code,
@@ -986,6 +990,14 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
}
break;
case CSI_SINK_PAD:
+ crop.left = 0;
+ crop.top = 0;
+ crop.width = sdformat->format.width;
+ crop.height = sdformat->format.height;
+ ret = csi_try_crop(priv, &crop, sensor);
+ if (ret)
+ goto out;
+
cc = imx_media_find_format(0, sdformat->format.code,
true, false);
if (!cc) {
@@ -1004,9 +1016,8 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
} else {
priv->format_mbus[sdformat->pad] = sdformat->format;
priv->cc[sdformat->pad] = cc;
- /* Update the crop window if this is an output pad */
- if (sdformat->pad == CSI_SRC_PAD_DIRECT ||
- sdformat->pad == CSI_SRC_PAD_IDMAC)
+ /* Reset the crop window if this is the input pad */
+ if (sdformat->pad == CSI_SINK_PAD)
priv->crop = crop;
}
--
2.7.4
The cropping rectangle was being modified by the output pad's
set_fmt, which is the wrong pad to do this. The crop rectangle
should not be modified by the output pad set_fmt. It instead
should be reset to the full input frame when the input pad format
is set.
The output pad set_fmt should set width/height to the current
crop dimensions, or 1/2 the crop width/height to enable
downscaling.
So the other part of this patch is to enable downscaling if
the output pad dimension(s) are 1/2 the crop dimension(s) at
csi_setup() time.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
drivers/staging/media/imx/imx-media-csi.c | 35 ++++++++++++++++++++-----------
1 file changed, 23 insertions(+), 12 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index a7d04e4..577038e 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -536,6 +536,10 @@ static int csi_setup(struct csi_priv *priv)
ipu_csi_set_window(priv->csi, &priv->crop);
+ ipu_csi_set_downsize(priv->csi,
+ priv->crop.width == 2 * outfmt->width,
+ priv->crop.height == 2 * outfmt->height);
+
ipu_csi_init_interface(priv->csi, &sensor_mbus_cfg, &if_fmt);
ipu_csi_set_dest(priv->csi, priv->dest);
@@ -932,15 +936,15 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
switch (sdformat->pad) {
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
- crop.left = priv->crop.left;
- crop.top = priv->crop.top;
- crop.width = sdformat->format.width;
- crop.height = sdformat->format.height;
- ret = csi_try_crop(priv, &crop, sensor);
- if (ret)
- goto out;
- sdformat->format.width = crop.width;
- sdformat->format.height = crop.height;
+ if (sdformat->format.width < priv->crop.width * 3 / 4)
+ sdformat->format.width = priv->crop.width / 2;
+ else
+ sdformat->format.width = priv->crop.width;
+
+ if (sdformat->format.height < priv->crop.height * 3 / 4)
+ sdformat->format.height = priv->crop.height / 2;
+ else
+ sdformat->format.height = priv->crop.height;
if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
cc = imx_media_find_format(0, sdformat->format.code,
@@ -986,6 +990,14 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
}
break;
case CSI_SINK_PAD:
+ crop.left = 0;
+ crop.top = 0;
+ crop.width = sdformat->format.width;
+ crop.height = sdformat->format.height;
+ ret = csi_try_crop(priv, &crop, sensor);
+ if (ret)
+ goto out;
+
cc = imx_media_find_format(0, sdformat->format.code,
true, false);
if (!cc) {
@@ -1004,9 +1016,8 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
} else {
priv->format_mbus[sdformat->pad] = sdformat->format;
priv->cc[sdformat->pad] = cc;
- /* Update the crop window if this is an output pad */
- if (sdformat->pad == CSI_SRC_PAD_DIRECT ||
- sdformat->pad == CSI_SRC_PAD_IDMAC)
+ /* Reset the crop window if this is the input pad */
+ if (sdformat->pad == CSI_SINK_PAD)
priv->crop = crop;
}
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
This is a set of three media entity subdevice drivers for the i.MX
Image Converter:
- Pre-process Router: Takes input frames from CSI0, CSI1, or VDIC.
Two output pads enable either or both of the preprocess tasks
below. If the input is from one of the CSIs, both proprocess task
links can be enabled to process frames from that CSI simultaneously.
If the input is the VDIC, only the Pre-processing Viewfinder task
link can be enabled.
- Pre-processing Encode task: video frames are routed directly from
the CSI and can be scaled, color-space converted, and rotated.
Scaled output is limited to 1024x1024 resolution. Output frames
are routed to the capture device.
- Pre-processing Viewfinder task: this task can perform the same
conversions as the pre-process encode task, but in addition can
be used for hardware motion compensated deinterlacing. Frames can
come either directly from the CSI or from the VDIC. Scaled output
is limited to 1024x1024 resolution. Output frames are routed to
the capture device.
drivers/staging/media/imx/Makefile | 2 +
drivers/staging/media/imx/imx-ic-common.c | 113 +++
drivers/staging/media/imx/imx-ic-prp.c | 475 +++++++++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 1219 +++++++++++++++++++++++++++
drivers/staging/media/imx/imx-ic.h | 38 +
5 files changed, 1847 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-ic-common.c
create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
create mode 100644 drivers/staging/media/imx/imx-ic.h
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 1f01520..878a126 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -1,9 +1,11 @@
imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o
imx-media-common-objs := imx-media-utils.o imx-media-fim.o
+imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-ic-common.c b/drivers/staging/media/imx/imx-ic-common.c
new file mode 100644
index 0000000..cfdd490
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -0,0 +1,113 @@
+/*
+ * V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+#define IC_TASK_PRP IC_NUM_TASKS
+#define IC_NUM_OPS (IC_NUM_TASKS + 1)
+
+static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
+ [IC_TASK_PRP] = &imx_ic_prp_ops,
+ [IC_TASK_ENCODER] = &imx_ic_prpencvf_ops,
+ [IC_TASK_VIEWFINDER] = &imx_ic_prpencvf_ops,
+};
+
+static int imx_ic_probe(struct platform_device *pdev)
+{
+ struct imx_media_internal_sd_platformdata *pdata;
+ struct imx_ic_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ /* get our ipu_id, grp_id and IC task id */
+ pdata = priv->dev->platform_data;
+ priv->ipu_id = pdata->ipu_id;
+ switch (pdata->grp_id) {
+ case IMX_MEDIA_GRP_ID_IC_PRP:
+ priv->task_id = IC_TASK_PRP;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRPENC:
+ priv->task_id = IC_TASK_ENCODER;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRPVF:
+ priv->task_id = IC_TASK_VIEWFINDER;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ v4l2_subdev_init(&priv->sd, ic_ops[priv->task_id]->subdev_ops);
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = ic_ops[priv->task_id]->internal_ops;
+ priv->sd.entity.ops = ic_ops[priv->task_id]->entity_ops;
+ priv->sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+ priv->sd.dev = &pdev->dev;
+ priv->sd.owner = THIS_MODULE;
+ priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ priv->sd.grp_id = pdata->grp_id;
+ strncpy(priv->sd.name, pdata->sd_name, sizeof(priv->sd.name));
+
+ ret = ic_ops[priv->task_id]->init(priv);
+ if (ret)
+ return ret;
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ ic_ops[priv->task_id]->remove(priv);
+
+ return ret;
+}
+
+static int imx_ic_remove(struct platform_device *pdev)
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct imx_ic_priv *priv = container_of(sd, struct imx_ic_priv, sd);
+
+ v4l2_info(sd, "Removing\n");
+
+ ic_ops[priv->task_id]->remove(priv);
+
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct platform_device_id imx_ic_ids[] = {
+ { .name = "imx-ipuv3-ic" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_ic_ids);
+
+static struct platform_driver imx_ic_driver = {
+ .probe = imx_ic_probe,
+ .remove = imx_ic_remove,
+ .id_table = imx_ic_ids,
+ .driver = {
+ .name = "imx-ipuv3-ic",
+ },
+};
+module_platform_driver(imx_ic_driver);
+
+MODULE_DESCRIPTION("i.MX IC subdev driver");
+MODULE_AUTHOR("Steve Longerbeam <steve_lo...@mentor.com>");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:imx-ipuv3-ic");
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
new file mode 100644
index 0000000..1832915
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -0,0 +1,475 @@
+/*
+ * V4L2 Capture IC Preprocess Subdev for Freescale i.MX5/6 SOC
+ *
+ * This subdevice handles capture of video frames from the CSI or VDIC,
+ * which are routed directly to the Image Converter preprocess tasks,
+ * for resizing, colorspace conversion, and rotation.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+/*
+ * Min/Max supported width and heights.
+ */
+#define MIN_W 176
+#define MIN_H 144
+#define MAX_W 4096
+#define MAX_H 4096
+#define W_ALIGN 4 /* multiple of 16 pixels */
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct prp_priv {
+ struct imx_media_dev *md;
+ struct imx_ic_priv *ic_priv;
+ struct media_pad pad[PRP_NUM_PADS];
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* IPU units we require */
+ struct ipu_soc *ipu;
+
+ struct v4l2_subdev *src_sd;
+ struct v4l2_subdev *sink_sd_prpenc;
+ struct v4l2_subdev *sink_sd_prpvf;
+
+ /* the CSI id at link validate */
+ int csi_id;
+
+ struct v4l2_mbus_framefmt format_mbus[PRP_NUM_PADS];
+ const struct imx_media_pixfmt *cc[PRP_NUM_PADS];
+ struct v4l2_fract frame_interval;
+
+ bool stream_on; /* streaming is on */
+};
+
+static inline struct prp_priv *sd_to_priv(struct v4l2_subdev *sd)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+
+ return ic_priv->prp_priv;
+}
+
+static int prp_start(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ priv->ipu = priv->md->ipu[ic_priv->ipu_id];
+
+ /* set IC to receive from CSI or VDI depending on source */
+ if (priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC)
+ ipu_set_ic_src_mux(priv->ipu, 0, true);
+ else
+ ipu_set_ic_src_mux(priv->ipu, priv->csi_id, false);
+
+ return 0;
+}
+
+static void prp_stop(struct prp_priv *priv)
+{
+}
+
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
+}
+
+static struct v4l2_mbus_framefmt *
+__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&ic_priv->sd, cfg, pad);
+ else
+ return &priv->format_mbus[pad];
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int prp_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_mbus_framefmt *fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+ if (!fmt) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ sdformat->format = *fmt;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ const struct imx_media_pixfmt *cc = NULL;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ cc = imx_media_find_ipu_format(0, sdformat->format.code, true);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ cc = imx_media_find_ipu_format(0, code, true);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+ /* Output pads mirror input pad */
+ if (sdformat->pad == PRP_SRC_PAD_PRPENC ||
+ sdformat->pad == PRP_SRC_PAD_PRPVF) {
+ infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD,
+ sdformat->which);
+ sdformat->format = *infmt;
+ }
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_s_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad != PRP_SINK_PAD)
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(ic_priv->dev, "link setup %s -> %s", remote->entity->name,
+ local->entity->name);
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ mutex_lock(&priv->lock);
+
+ if (local->flags & MEDIA_PAD_FL_SINK) {
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ if (priv->sink_sd_prpenc && (remote_sd->grp_id &
+ IMX_MEDIA_GRP_ID_VDIC)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is a source pad */
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ switch (local->index) {
+ case PRP_SRC_PAD_PRPENC:
+ if (priv->sink_sd_prpenc) {
+ ret = -EBUSY;
+ goto out;
+ }
+ if (priv->src_sd && (priv->src_sd->grp_id &
+ IMX_MEDIA_GRP_ID_VDIC)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ priv->sink_sd_prpenc = remote_sd;
+ break;
+ case PRP_SRC_PAD_PRPVF:
+ if (priv->sink_sd_prpvf) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->sink_sd_prpvf = remote_sd;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ } else {
+ switch (local->index) {
+ case PRP_SRC_PAD_PRPENC:
+ priv->sink_sd_prpenc = NULL;
+ break;
+ case PRP_SRC_PAD_PRPVF:
+ priv->sink_sd_prpvf = NULL;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_link_validate(struct v4l2_subdev *sd,
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ struct imx_media_subdev *csi;
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+ if (ret)
+ return ret;
+
+ csi = imx_media_find_pipeline_subdev(priv->md, &ic_priv->sd.entity,
+ IMX_MEDIA_GRP_ID_CSI);
+ if (IS_ERR(csi))
+ csi = NULL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC) {
+ /*
+ * the ->PRPENC link cannot be enabled if the source
+ * is the VDIC
+ */
+ if (priv->sink_sd_prpenc)
+ ret = -EINVAL;
+ goto out;
+ } else {
+ /* the source is a CSI */
+ if (!csi) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ switch (csi->sd->grp_id) {
+ case IMX_MEDIA_GRP_ID_CSI0:
+ priv->csi_id = 0;
+ break;
+ case IMX_MEDIA_GRP_ID_CSI1:
+ priv->csi_id = 1;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->src_sd || (!priv->sink_sd_prpenc && !priv->sink_sd_prpvf)) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ dev_dbg(ic_priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+
+ if (enable && !priv->stream_on)
+ ret = prp_start(priv);
+ else if (!enable && priv->stream_on)
+ prp_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int prp_registered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ int i, ret;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ for (i = 0; i < PRP_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == PRP_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ /* set a default mbus format */
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ return ret;
+ }
+
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+ return media_entity_pads_init(&sd->entity, PRP_NUM_PADS, priv->pad);
+}
+
+static struct v4l2_subdev_pad_ops prp_pad_ops = {
+ .enum_mbus_code = prp_enum_mbus_code,
+ .get_fmt = prp_get_fmt,
+ .set_fmt = prp_set_fmt,
+ .link_validate = prp_link_validate,
+};
+
+static struct v4l2_subdev_video_ops prp_video_ops = {
+ .s_stream = prp_s_stream,
+ .g_frame_interval = prp_g_frame_interval,
+ .s_frame_interval = prp_s_frame_interval,
+};
+
+static struct media_entity_operations prp_entity_ops = {
+ .link_setup = prp_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_ops prp_subdev_ops = {
+ .video = &prp_video_ops,
+ .pad = &prp_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops prp_internal_ops = {
+ .registered = prp_registered,
+};
+
+static int prp_init(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv;
+
+ priv = devm_kzalloc(ic_priv->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ mutex_init(&priv->lock);
+ ic_priv->prp_priv = priv;
+ priv->ic_priv = ic_priv;
+
+ return 0;
+}
+
+static void prp_remove(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv = ic_priv->prp_priv;
+
+ mutex_destroy(&priv->lock);
+}
+
+struct imx_ic_ops imx_ic_prp_ops = {
+ .subdev_ops = &prp_subdev_ops,
+ .internal_ops = &prp_internal_ops,
+ .entity_ops = &prp_entity_ops,
+ .init = prp_init,
+ .remove = prp_remove,
+};
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
new file mode 100644
index 0000000..bd89e9b
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -0,0 +1,1219 @@
+/*
+ * V4L2 Capture IC Preprocess Subdev for Freescale i.MX5/6 SOC
+ *
+ * This subdevice handles capture of video frames from the CSI or VDIC,
+ * which are routed directly to the Image Converter preprocess tasks,
+ * for resizing, colorspace conversion, and rotation.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+/*
+ * Min/Max supported width and heights.
+ *
+ * We allow planar output, so we have to align width at the source pad
+ * by 16 pixels to meet IDMAC alignment requirements for possible planar
+ * output.
+ *
+ * TODO: move this into pad format negotiation, if capture device
+ * has not requested a planar format, we should allow 8 pixel
+ * alignment at the source pad.
+ */
+#define MIN_W_SINK 176
+#define MIN_H_SINK 144
+#define MAX_W_SINK 4096
+#define MAX_H_SINK 4096
+#define W_ALIGN_SINK 3 /* multiple of 8 pixels */
+#define H_ALIGN_SINK 1 /* multiple of 2 lines */
+
+#define MAX_W_SRC 1024
+#define MAX_H_SRC 1024
+#define W_ALIGN_SRC 4 /* multiple of 16 pixels */
+#define H_ALIGN_SRC 1 /* multiple of 2 lines */
+
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct prp_priv {
+ struct imx_media_dev *md;
+ struct imx_ic_priv *ic_priv;
+ struct media_pad pad[PRPENCVF_NUM_PADS];
+ /* the video device at output pad */
+ struct imx_media_video_dev *vdev;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* IPU units we require */
+ struct ipu_soc *ipu;
+ struct ipu_ic *ic;
+ struct ipuv3_channel *out_ch;
+ struct ipuv3_channel *rot_in_ch;
+ struct ipuv3_channel *rot_out_ch;
+
+ /* active vb2 buffers to send to video dev sink */
+ struct imx_media_buffer *active_vb2_buf[2];
+ struct imx_media_dma_buf underrun_buf;
+
+ int ipu_buf_num; /* ipu double buffer index: 0-1 */
+
+ /* the sink for the captured frames */
+ struct media_entity *sink;
+ /* the source subdev */
+ struct v4l2_subdev *src_sd;
+
+ /* the attached CSI at stream on */
+ struct v4l2_subdev *csi_sd;
+
+ struct v4l2_mbus_framefmt format_mbus[PRPENCVF_NUM_PADS];
+ const struct imx_media_pixfmt *cc[PRPENCVF_NUM_PADS];
+ struct v4l2_fract frame_interval;
+
+ struct imx_media_dma_buf rot_buf[2];
+
+ /* controls */
+ struct v4l2_ctrl_handler ctrl_hdlr;
+ int rotation; /* degrees */
+ bool hflip;
+ bool vflip;
+
+ /* derived from rotation, hflip, vflip controls */
+ enum ipu_rotate_mode rot_mode;
+
+ spinlock_t irqlock; /* protect eof_irq handler */
+
+ struct timer_list eof_timeout_timer;
+ int eof_irq;
+ int nfb4eof_irq;
+
+ bool stream_on; /* streaming is on */
+ bool last_eof; /* waiting for last EOF at stream off */
+ struct completion last_eof_comp;
+};
+
+static const struct prp_channels {
+ u32 out_ch;
+ u32 rot_in_ch;
+ u32 rot_out_ch;
+} prp_channel[] = {
+ [IC_TASK_ENCODER] = {
+ .out_ch = IPUV3_CHANNEL_IC_PRP_ENC_MEM,
+ .rot_in_ch = IPUV3_CHANNEL_MEM_ROT_ENC,
+ .rot_out_ch = IPUV3_CHANNEL_ROT_ENC_MEM,
+ },
+ [IC_TASK_VIEWFINDER] = {
+ .out_ch = IPUV3_CHANNEL_IC_PRP_VF_MEM,
+ .rot_in_ch = IPUV3_CHANNEL_MEM_ROT_VF,
+ .rot_out_ch = IPUV3_CHANNEL_ROT_VF_MEM,
+ },
+};
+
+static inline struct prp_priv *sd_to_priv(struct v4l2_subdev *sd)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+
+ return ic_priv->task_priv;
+}
+
+static void prp_put_ipu_resources(struct prp_priv *priv)
+{
+ if (!IS_ERR_OR_NULL(priv->ic))
+ ipu_ic_put(priv->ic);
+ priv->ic = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->out_ch))
+ ipu_idmac_put(priv->out_ch);
+ priv->out_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->rot_in_ch))
+ ipu_idmac_put(priv->rot_in_ch);
+ priv->rot_in_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->rot_out_ch))
+ ipu_idmac_put(priv->rot_out_ch);
+ priv->rot_out_ch = NULL;
+}
+
+static int prp_get_ipu_resources(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ int ret, task = ic_priv->task_id;
+
+ priv->ipu = priv->md->ipu[ic_priv->ipu_id];
+
+ priv->ic = ipu_ic_get(priv->ipu, task);
+ if (IS_ERR(priv->ic)) {
+ v4l2_err(&ic_priv->sd, "failed to get IC\n");
+ ret = PTR_ERR(priv->ic);
+ goto out;
+ }
+
+ priv->out_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].out_ch);
+ if (IS_ERR(priv->out_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].out_ch);
+ ret = PTR_ERR(priv->out_ch);
+ goto out;
+ }
+
+ priv->rot_in_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].rot_in_ch);
+ if (IS_ERR(priv->rot_in_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].rot_in_ch);
+ ret = PTR_ERR(priv->rot_in_ch);
+ goto out;
+ }
+
+ priv->rot_out_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].rot_out_ch);
+ if (IS_ERR(priv->rot_out_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].rot_out_ch);
+ ret = PTR_ERR(priv->rot_out_ch);
+ goto out;
+ }
+
+ return 0;
+out:
+ prp_put_ipu_resources(priv);
+ return ret;
+}
+
+static void prp_vb2_buf_done(struct prp_priv *priv, struct ipuv3_channel *ch)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *done, *next;
+ struct vb2_buffer *vb;
+ dma_addr_t phys;
+
+ done = priv->active_vb2_buf[priv->ipu_buf_num];
+ if (done) {
+ vb = &done->vbuf.vb2_buf;
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+ }
+
+ /* get next queued buffer */
+ next = imx_media_capture_device_next_buf(vdev);
+ if (next) {
+ phys = vb2_dma_contig_plane_dma_addr(&next->vbuf.vb2_buf, 0);
+ priv->active_vb2_buf[priv->ipu_buf_num] = next;
+ } else {
+ phys = priv->underrun_buf.phys;
+ priv->active_vb2_buf[priv->ipu_buf_num] = NULL;
+ }
+
+ if (ipu_idmac_buffer_is_ready(ch, priv->ipu_buf_num))
+ ipu_idmac_clear_buffer(ch, priv->ipu_buf_num);
+
+ ipu_cpmem_set_buffer(ch, priv->ipu_buf_num, phys);
+}
+
+static irqreturn_t prp_eof_interrupt(int irq, void *dev_id)
+{
+ struct prp_priv *priv = dev_id;
+ struct ipuv3_channel *channel;
+
+ spin_lock(&priv->irqlock);
+
+ if (priv->last_eof) {
+ complete(&priv->last_eof_comp);
+ priv->last_eof = false;
+ goto unlock;
+ }
+
+ /*
+ * if there is a CSI in the pipeline, inform it of this EOF
+ * so it can monitor frame intervals.
+ */
+ if (priv->csi_sd)
+ v4l2_subdev_call(priv->csi_sd, core,
+ interrupt_service_routine,
+ 0, NULL);
+
+ channel = (ipu_rot_mode_is_irt(priv->rot_mode)) ?
+ priv->rot_out_ch : priv->out_ch;
+
+ prp_vb2_buf_done(priv, channel);
+
+ /* select new IPU buf */
+ ipu_idmac_select_buffer(channel, priv->ipu_buf_num);
+ /* toggle IPU double-buffer index */
+ priv->ipu_buf_num ^= 1;
+
+ /* bump the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+unlock:
+ spin_unlock(&priv->irqlock);
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t prp_nfb4eof_interrupt(int irq, void *dev_id)
+{
+ struct prp_priv *priv = dev_id;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ static const struct v4l2_event ev = {
+ .type = V4L2_EVENT_NEW_FRAME_BEFORE_EOF,
+ };
+
+ v4l2_err(&ic_priv->sd, "NFB4EOF\n");
+
+ v4l2_subdev_notify_event(&ic_priv->sd, &ev);
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * EOF timeout timer function.
+ */
+/*
+ * EOF timeout timer function. This is an unrecoverable condition
+ * without a stream restart.
+ */
+static void prp_eof_timeout(unsigned long data)
+{
+ struct prp_priv *priv = (struct prp_priv *)data;
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ v4l2_err(&ic_priv->sd, "EOF timeout\n");
+
+ /* signal a fatal error to capture device */
+ imx_media_capture_device_error(vdev);
+}
+
+static void prp_setup_vb2_buf(struct prp_priv *priv, dma_addr_t *phys)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *buf;
+ int i;
+
+ for (i = 0; i < 2; i++) {
+ buf = imx_media_capture_device_next_buf(vdev);
+ priv->active_vb2_buf[i] = buf;
+ phys[i] = vb2_dma_contig_plane_dma_addr(&buf->vbuf.vb2_buf, 0);
+ }
+}
+
+static void prp_unsetup_vb2_buf(struct prp_priv *priv,
+ enum vb2_buffer_state return_state)
+{
+ struct imx_media_buffer *buf;
+ int i;
+
+ /* return any remaining active frames with error */
+ for (i = 0; i < 2; i++) {
+ buf = priv->active_vb2_buf[i];
+ if (buf) {
+ struct vb2_buffer *vb = &buf->vbuf.vb2_buf;
+
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, return_state);
+ }
+ }
+}
+
+static int prp_setup_channel(struct prp_priv *priv,
+ struct ipuv3_channel *channel,
+ enum ipu_rotate_mode rot_mode,
+ dma_addr_t addr0, dma_addr_t addr1,
+ bool rot_swap_width_height)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ const struct imx_media_pixfmt *outcc;
+ struct v4l2_mbus_framefmt *infmt;
+ unsigned int burst_size;
+ struct ipu_image image;
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ipu_cpmem_zero(channel);
+
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ image.rect.width = image.pix.width;
+ image.rect.height = image.pix.height;
+
+ if (rot_swap_width_height) {
+ swap(image.pix.width, image.pix.height);
+ swap(image.rect.width, image.rect.height);
+ /* recalc stride using swapped width */
+ image.pix.bytesperline = outcc->planar ?
+ image.pix.width :
+ (image.pix.width * outcc->bpp) >> 3;
+ }
+
+ image.phys0 = addr0;
+ image.phys1 = addr1;
+
+ ret = ipu_cpmem_set_image(channel, &image);
+ if (ret)
+ return ret;
+
+ if (channel == priv->rot_in_ch ||
+ channel == priv->rot_out_ch) {
+ burst_size = 8;
+ ipu_cpmem_set_block_mode(channel);
+ } else {
+ burst_size = (image.pix.width & 0xf) ? 8 : 16;
+ }
+
+ ipu_cpmem_set_burstsize(channel, burst_size);
+
+ if (rot_mode)
+ ipu_cpmem_set_rotation(channel, rot_mode);
+
+ if (image.pix.field == V4L2_FIELD_NONE &&
+ V4L2_FIELD_HAS_BOTH(infmt->field) &&
+ channel == priv->out_ch)
+ ipu_cpmem_interlaced_scan(channel, image.pix.bytesperline);
+
+ ret = ipu_ic_task_idma_init(priv->ic, channel,
+ image.pix.width, image.pix.height,
+ burst_size, rot_mode);
+ if (ret)
+ return ret;
+
+ ipu_cpmem_set_axi_id(channel, 1);
+
+ ipu_idmac_set_double_buffer(channel, true);
+
+ return 0;
+}
+
+static int prp_setup_rotation(struct prp_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ const struct imx_media_pixfmt *outcc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct v4l2_pix_format *outfmt;
+ dma_addr_t phys[2];
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outfmt = &vdev->fmt.fmt.pix;
+ incc = priv->cc[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->rot_buf[0],
+ outfmt->sizeimage);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "failed to alloc rot_buf[0], %d\n", ret);
+ return ret;
+ }
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->rot_buf[1],
+ outfmt->sizeimage);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "failed to alloc rot_buf[1], %d\n", ret);
+ goto free_rot0;
+ }
+
+ ret = ipu_ic_task_init(priv->ic,
+ infmt->width, infmt->height,
+ outfmt->height, outfmt->width,
+ incc->cs, outcc->cs);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "ipu_ic_task_init failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* init the IC-PRP-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->out_ch, IPU_ROTATE_NONE,
+ priv->rot_buf[0].phys, priv->rot_buf[1].phys,
+ true);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(out_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* init the MEM-->IC-PRP ROT IDMAC channel */
+ ret = prp_setup_channel(priv, priv->rot_in_ch, priv->rot_mode,
+ priv->rot_buf[0].phys, priv->rot_buf[1].phys,
+ true);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(rot_in_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ prp_setup_vb2_buf(priv, phys);
+
+ /* init the destination IC-PRP ROT-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->rot_out_ch, IPU_ROTATE_NONE,
+ phys[0], phys[1],
+ false);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(rot_out_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* now link IC-PRP-->MEM to MEM-->IC-PRP ROT */
+ ipu_idmac_link(priv->out_ch, priv->rot_in_ch);
+
+ /* enable the IC */
+ ipu_ic_enable(priv->ic);
+
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->out_ch, 0);
+ ipu_idmac_select_buffer(priv->out_ch, 1);
+ ipu_idmac_select_buffer(priv->rot_out_ch, 0);
+ ipu_idmac_select_buffer(priv->rot_out_ch, 1);
+
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->out_ch);
+ ipu_idmac_enable_channel(priv->rot_in_ch);
+ ipu_idmac_enable_channel(priv->rot_out_ch);
+
+ /* and finally enable the IC PRP task */
+ ipu_ic_task_enable(priv->ic);
+
+ return 0;
+
+free_rot1:
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[1]);
+free_rot0:
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[0]);
+ return ret;
+}
+
+static void prp_unsetup_rotation(struct prp_priv *priv)
+{
+ ipu_ic_task_disable(priv->ic);
+
+ ipu_idmac_disable_channel(priv->out_ch);
+ ipu_idmac_disable_channel(priv->rot_in_ch);
+ ipu_idmac_disable_channel(priv->rot_out_ch);
+
+ ipu_idmac_unlink(priv->out_ch, priv->rot_in_ch);
+
+ ipu_ic_disable(priv->ic);
+
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[0]);
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[1]);
+}
+
+static int prp_setup_norotation(struct prp_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ const struct imx_media_pixfmt *outcc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct v4l2_pix_format *outfmt;
+ dma_addr_t phys[2];
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outfmt = &vdev->fmt.fmt.pix;
+ incc = priv->cc[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ret = ipu_ic_task_init(priv->ic,
+ infmt->width, infmt->height,
+ outfmt->width, outfmt->height,
+ incc->cs, outcc->cs);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "ipu_ic_task_init failed, %d\n", ret);
+ return ret;
+ }
+
+ prp_setup_vb2_buf(priv, phys);
+
+ /* init the IC PRP-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->out_ch, priv->rot_mode,
+ phys[0], phys[1], false);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(out_ch) failed, %d\n", ret);
+ return ret;
+ }
+
+ ipu_cpmem_dump(priv->out_ch);
+ ipu_ic_dump(priv->ic);
+ ipu_dump(priv->ipu);
+
+ ipu_ic_enable(priv->ic);
+
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->out_ch, 0);
+ ipu_idmac_select_buffer(priv->out_ch, 1);
+
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->out_ch);
+
+ /* enable the IC task */
+ ipu_ic_task_enable(priv->ic);
+
+ return 0;
+}
+
+static void prp_unsetup_norotation(struct prp_priv *priv)
+{
+ ipu_ic_task_disable(priv->ic);
+ ipu_idmac_disable_channel(priv->out_ch);
+ ipu_ic_disable(priv->ic);
+}
+
+static void prp_unsetup(struct prp_priv *priv,
+ enum vb2_buffer_state state)
+{
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ prp_unsetup_rotation(priv);
+ else
+ prp_unsetup_norotation(priv);
+
+ prp_unsetup_vb2_buf(priv, state);
+}
+
+static int prp_start(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_pix_format *outfmt;
+ int ret;
+
+ ret = prp_get_ipu_resources(priv);
+ if (ret)
+ return ret;
+
+ outfmt = &vdev->fmt.fmt.pix;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->underrun_buf,
+ outfmt->sizeimage);
+ if (ret)
+ goto out_put_ipu;
+
+ priv->ipu_buf_num = 0;
+
+ /* init EOF completion waitq */
+ init_completion(&priv->last_eof_comp);
+ priv->last_eof = false;
+
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ ret = prp_setup_rotation(priv);
+ else
+ ret = prp_setup_norotation(priv);
+ if (ret)
+ goto out_free_underrun;
+
+ priv->nfb4eof_irq = ipu_idmac_channel_irq(priv->ipu,
+ priv->out_ch,
+ IPU_IRQ_NFB4EOF);
+ ret = devm_request_irq(ic_priv->dev, priv->nfb4eof_irq,
+ prp_nfb4eof_interrupt, 0,
+ "imx-ic-prp-nfb4eof", priv);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "Error registering NFB4EOF irq: %d\n", ret);
+ goto out_unsetup;
+ }
+
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ priv->eof_irq = ipu_idmac_channel_irq(
+ priv->ipu, priv->rot_out_ch, IPU_IRQ_EOF);
+ else
+ priv->eof_irq = ipu_idmac_channel_irq(
+ priv->ipu, priv->out_ch, IPU_IRQ_EOF);
+
+ ret = devm_request_irq(ic_priv->dev, priv->eof_irq,
+ prp_eof_interrupt, 0,
+ "imx-ic-prp-eof", priv);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "Error registering eof irq: %d\n", ret);
+ goto out_free_nfb4eof_irq;
+ }
+
+ /* start the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+ return 0;
+
+out_free_nfb4eof_irq:
+ devm_free_irq(ic_priv->dev, priv->nfb4eof_irq, priv);
+out_unsetup:
+ prp_unsetup(priv, VB2_BUF_STATE_QUEUED);
+out_free_underrun:
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+out_put_ipu:
+ prp_put_ipu_resources(priv);
+ return ret;
+}
+
+static void prp_stop(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ unsigned long flags;
+ int ret;
+
+ /* mark next EOF interrupt as the last before stream off */
+ spin_lock_irqsave(&priv->irqlock, flags);
+ priv->last_eof = true;
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+
+ /*
+ * and then wait for interrupt handler to mark completion.
+ */
+ ret = wait_for_completion_timeout(
+ &priv->last_eof_comp,
+ msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+ if (ret == 0)
+ v4l2_warn(&ic_priv->sd, "wait last EOF timeout\n");
+
+ devm_free_irq(ic_priv->dev, priv->eof_irq, priv);
+ devm_free_irq(ic_priv->dev, priv->nfb4eof_irq, priv);
+
+ prp_unsetup(priv, VB2_BUF_STATE_ERROR);
+
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+
+ /* cancel the EOF timeout timer */
+ del_timer_sync(&priv->eof_timeout_timer);
+
+ prp_put_ipu_resources(priv);
+}
+
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ if (code->pad == PRPENCVF_SRC_PAD)
+ return imx_media_enum_format(NULL, &code->code, code->index,
+ true, false);
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
+}
+
+static struct v4l2_mbus_framefmt *
+__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&ic_priv->sd, cfg, pad);
+ else
+ return &priv->format_mbus[pad];
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int prp_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_mbus_framefmt *fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+ if (!fmt) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ sdformat->format = *fmt;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ const struct imx_media_pixfmt *cc;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (sdformat->pad == PRPENCVF_SRC_PAD) {
+ infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
+ sdformat->which);
+
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0, true, false);
+ cc = imx_media_find_format(0, code, true, false);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ if (sdformat->format.field != V4L2_FIELD_NONE)
+ sdformat->format.field = infmt->field;
+
+ /* IC resizer cannot downsize more than 4:1 */
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ v4l_bound_align_image(&sdformat->format.width,
+ infmt->height / 4, MAX_H_SRC,
+ H_ALIGN_SRC,
+ &sdformat->format.height,
+ infmt->width / 4, MAX_W_SRC,
+ W_ALIGN_SRC, S_ALIGN);
+ else
+ v4l_bound_align_image(&sdformat->format.width,
+ infmt->width / 4, MAX_W_SRC,
+ W_ALIGN_SRC,
+ &sdformat->format.height,
+ infmt->height / 4, MAX_H_SRC,
+ H_ALIGN_SRC, S_ALIGN);
+ } else {
+ cc = imx_media_find_ipu_format(0, sdformat->format.code,
+ true);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ cc = imx_media_find_ipu_format(0, code, true);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ v4l_bound_align_image(&sdformat->format.width,
+ MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
+ &sdformat->format.height,
+ MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
+ S_ALIGN);
+ }
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_s_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad == PRPENCVF_SRC_PAD)
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(ic_priv->dev, "link setup %s -> %s", remote->entity->name,
+ local->entity->name);
+
+ mutex_lock(&priv->lock);
+
+ if (local->flags & MEDIA_PAD_FL_SINK) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is the source pad */
+
+ /* the remote must be the device node */
+ if (!is_media_entity_v4l2_video_device(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->sink) {
+ ret = -EBUSY;
+ goto out;
+ }
+ } else {
+ priv->sink = NULL;
+ goto out;
+ }
+
+ priv->sink = remote->entity;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_link_validate(struct v4l2_subdev *sd,
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ struct imx_media_subdev *csi;
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+ if (ret)
+ return ret;
+
+ csi = imx_media_find_pipeline_subdev(priv->md, &ic_priv->sd.entity,
+ IMX_MEDIA_GRP_ID_CSI);
+ if (IS_ERR(csi))
+ csi = NULL;
+
+ mutex_lock(&priv->lock);
+ priv->csi_sd = csi ? csi->sd : NULL;
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct prp_priv *priv = container_of(ctrl->handler,
+ struct prp_priv, ctrl_hdlr);
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ enum ipu_rotate_mode rot_mode;
+ int rotation, ret = 0;
+ bool hflip, vflip;
+
+ mutex_lock(&priv->lock);
+
+ rotation = priv->rotation;
+ hflip = priv->hflip;
+ vflip = priv->vflip;
+
+ switch (ctrl->id) {
+ case V4L2_CID_HFLIP:
+ hflip = (ctrl->val == 1);
+ break;
+ case V4L2_CID_VFLIP:
+ vflip = (ctrl->val == 1);
+ break;
+ case V4L2_CID_ROTATE:
+ rotation = ctrl->val;
+ break;
+ default:
+ v4l2_err(&ic_priv->sd, "Invalid control\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = ipu_degrees_to_rot_mode(&rot_mode, rotation, hflip, vflip);
+ /* can't change rotation mid-streaming */
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->rot_mode = rot_mode;
+ priv->rotation = rotation;
+ priv->hflip = hflip;
+ priv->vflip = vflip;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops prp_ctrl_ops = {
+ .s_ctrl = prp_s_ctrl,
+};
+
+static int prp_init_controls(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ struct v4l2_ctrl_handler *hdlr = &priv->ctrl_hdlr;
+ int ret;
+
+ v4l2_ctrl_handler_init(hdlr, 3);
+
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_HFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_VFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_ROTATE,
+ 0, 270, 90, 0);
+
+ ic_priv->sd.ctrl_handler = hdlr;
+
+ if (hdlr->error) {
+ ret = hdlr->error;
+ goto out_free;
+ }
+
+ v4l2_ctrl_handler_setup(hdlr);
+ return 0;
+
+out_free:
+ v4l2_ctrl_handler_free(hdlr);
+ return ret;
+}
+
+static int prp_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->src_sd || !priv->sink) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ dev_dbg(ic_priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+
+ if (enable && !priv->stream_on)
+ ret = prp_start(priv);
+ else if (!enable && priv->stream_on)
+ prp_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ if (sub->type != V4L2_EVENT_NEW_FRAME_BEFORE_EOF)
+ return -EINVAL;
+ if (sub->id != 0)
+ return -EINVAL;
+
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static int prp_unsubscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_unsubscribe(fh, sub);
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int prp_registered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ int i, ret;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
+ if (i == PRPENCVF_SINK_PAD) {
+ priv->pad[i].flags = MEDIA_PAD_FL_SINK;
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ } else {
+ priv->pad[i].flags = MEDIA_PAD_FL_SOURCE;
+ code = 0;
+ }
+
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ return ret;
+ }
+
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+ ret = media_entity_pads_init(&sd->entity, PRPENCVF_NUM_PADS,
+ priv->pad);
+ if (ret)
+ return ret;
+
+ ret = imx_media_capture_device_register(priv->vdev);
+ if (ret)
+ return ret;
+
+ ret = prp_init_controls(priv);
+ if (ret)
+ imx_media_capture_device_unregister(priv->vdev);
+
+ return ret;
+}
+
+static void prp_unregistered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ imx_media_capture_device_unregister(priv->vdev);
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+}
+
+static struct v4l2_subdev_pad_ops prp_pad_ops = {
+ .enum_mbus_code = prp_enum_mbus_code,
+ .get_fmt = prp_get_fmt,
+ .set_fmt = prp_set_fmt,
+ .link_validate = prp_link_validate,
+};
+
+static const struct v4l2_subdev_core_ops prp_core_ops = {
+ .subscribe_event = prp_subscribe_event,
+ .unsubscribe_event = prp_unsubscribe_event,
+};
+
+static struct v4l2_subdev_video_ops prp_video_ops = {
+ .s_stream = prp_s_stream,
+ .g_frame_interval = prp_g_frame_interval,
+ .s_frame_interval = prp_s_frame_interval,
+};
+
+static struct media_entity_operations prp_entity_ops = {
+ .link_setup = prp_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_ops prp_subdev_ops = {
+ .core = &prp_core_ops,
+ .video = &prp_video_ops,
+ .pad = &prp_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops prp_internal_ops = {
+ .registered = prp_registered,
+ .unregistered = prp_unregistered,
+};
+
+static int prp_init(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv;
+
+ priv = devm_kzalloc(ic_priv->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ ic_priv->task_priv = priv;
+ priv->ic_priv = ic_priv;
+
+ spin_lock_init(&priv->irqlock);
+ init_timer(&priv->eof_timeout_timer);
+ priv->eof_timeout_timer.data = (unsigned long)priv;
+ priv->eof_timeout_timer.function = prp_eof_timeout;
+
+ priv->vdev = imx_media_capture_device_init(&ic_priv->sd,
+ PRPENCVF_SRC_PAD);
+ if (IS_ERR(priv->vdev))
+ return PTR_ERR(priv->vdev);
+
+ mutex_init(&priv->lock);
+
+ return 0;
+}
+
+static void prp_remove(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv = ic_priv->task_priv;
+
+ mutex_destroy(&priv->lock);
+ imx_media_capture_device_remove(priv->vdev);
+}
+
+struct imx_ic_ops imx_ic_prpencvf_ops = {
+ .subdev_ops = &prp_subdev_ops,
+ .internal_ops = &prp_internal_ops,
+ .entity_ops = &prp_entity_ops,
+ .init = prp_init,
+ .remove = prp_remove,
+};
diff --git a/drivers/staging/media/imx/imx-ic.h b/drivers/staging/media/imx/imx-ic.h
new file mode 100644
index 0000000..5535111
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic.h
@@ -0,0 +1,38 @@
+/*
+ * V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#ifndef _IMX_IC_H
+#define _IMX_IC_H
+
+#include <media/v4l2-subdev.h>
+
+struct imx_ic_priv {
+ struct device *dev;
+ struct v4l2_subdev sd;
+ int ipu_id;
+ int task_id;
+ void *prp_priv;
+ void *task_priv;
+};
+
+struct imx_ic_ops {
+ struct v4l2_subdev_ops *subdev_ops;
+ struct v4l2_subdev_internal_ops *internal_ops;
+ struct media_entity_operations *entity_ops;
+
+ int (*init)(struct imx_ic_priv *ic_priv);
+ void (*remove)(struct imx_ic_priv *ic_priv);
+};
+
+extern struct imx_ic_ops imx_ic_prp_ops;
+extern struct imx_ic_ops imx_ic_prpencvf_ops;
+extern struct imx_ic_ops imx_ic_pp_ops;
+
+#endif
--
2.7.4
Image Converter:
- Pre-process Router: Takes input frames from CSI0, CSI1, or VDIC.
Two output pads enable either or both of the preprocess tasks
below. If the input is from one of the CSIs, both proprocess task
links can be enabled to process frames from that CSI simultaneously.
If the input is the VDIC, only the Pre-processing Viewfinder task
link can be enabled.
- Pre-processing Encode task: video frames are routed directly from
the CSI and can be scaled, color-space converted, and rotated.
Scaled output is limited to 1024x1024 resolution. Output frames
are routed to the capture device.
- Pre-processing Viewfinder task: this task can perform the same
conversions as the pre-process encode task, but in addition can
be used for hardware motion compensated deinterlacing. Frames can
come either directly from the CSI or from the VDIC. Scaled output
is limited to 1024x1024 resolution. Output frames are routed to
the capture device.
drivers/staging/media/imx/Makefile | 2 +
drivers/staging/media/imx/imx-ic-common.c | 113 +++
drivers/staging/media/imx/imx-ic-prp.c | 475 +++++++++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 1219 +++++++++++++++++++++++++++
drivers/staging/media/imx/imx-ic.h | 38 +
5 files changed, 1847 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-ic-common.c
create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
create mode 100644 drivers/staging/media/imx/imx-ic.h
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 1f01520..878a126 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -1,9 +1,11 @@
imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o
imx-media-common-objs := imx-media-utils.o imx-media-fim.o
+imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-ic-common.c b/drivers/staging/media/imx/imx-ic-common.c
new file mode 100644
index 0000000..cfdd490
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -0,0 +1,113 @@
+/*
+ * V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+#define IC_TASK_PRP IC_NUM_TASKS
+#define IC_NUM_OPS (IC_NUM_TASKS + 1)
+
+static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
+ [IC_TASK_PRP] = &imx_ic_prp_ops,
+ [IC_TASK_ENCODER] = &imx_ic_prpencvf_ops,
+ [IC_TASK_VIEWFINDER] = &imx_ic_prpencvf_ops,
+};
+
+static int imx_ic_probe(struct platform_device *pdev)
+{
+ struct imx_media_internal_sd_platformdata *pdata;
+ struct imx_ic_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ /* get our ipu_id, grp_id and IC task id */
+ pdata = priv->dev->platform_data;
+ priv->ipu_id = pdata->ipu_id;
+ switch (pdata->grp_id) {
+ case IMX_MEDIA_GRP_ID_IC_PRP:
+ priv->task_id = IC_TASK_PRP;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRPENC:
+ priv->task_id = IC_TASK_ENCODER;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRPVF:
+ priv->task_id = IC_TASK_VIEWFINDER;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ v4l2_subdev_init(&priv->sd, ic_ops[priv->task_id]->subdev_ops);
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = ic_ops[priv->task_id]->internal_ops;
+ priv->sd.entity.ops = ic_ops[priv->task_id]->entity_ops;
+ priv->sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+ priv->sd.dev = &pdev->dev;
+ priv->sd.owner = THIS_MODULE;
+ priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ priv->sd.grp_id = pdata->grp_id;
+ strncpy(priv->sd.name, pdata->sd_name, sizeof(priv->sd.name));
+
+ ret = ic_ops[priv->task_id]->init(priv);
+ if (ret)
+ return ret;
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ ic_ops[priv->task_id]->remove(priv);
+
+ return ret;
+}
+
+static int imx_ic_remove(struct platform_device *pdev)
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct imx_ic_priv *priv = container_of(sd, struct imx_ic_priv, sd);
+
+ v4l2_info(sd, "Removing\n");
+
+ ic_ops[priv->task_id]->remove(priv);
+
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct platform_device_id imx_ic_ids[] = {
+ { .name = "imx-ipuv3-ic" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_ic_ids);
+
+static struct platform_driver imx_ic_driver = {
+ .probe = imx_ic_probe,
+ .remove = imx_ic_remove,
+ .id_table = imx_ic_ids,
+ .driver = {
+ .name = "imx-ipuv3-ic",
+ },
+};
+module_platform_driver(imx_ic_driver);
+
+MODULE_DESCRIPTION("i.MX IC subdev driver");
+MODULE_AUTHOR("Steve Longerbeam <steve_lo...@mentor.com>");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:imx-ipuv3-ic");
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
new file mode 100644
index 0000000..1832915
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -0,0 +1,475 @@
+/*
+ * V4L2 Capture IC Preprocess Subdev for Freescale i.MX5/6 SOC
+ *
+ * This subdevice handles capture of video frames from the CSI or VDIC,
+ * which are routed directly to the Image Converter preprocess tasks,
+ * for resizing, colorspace conversion, and rotation.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+/*
+ * Min/Max supported width and heights.
+ */
+#define MIN_W 176
+#define MIN_H 144
+#define MAX_W 4096
+#define MAX_H 4096
+#define W_ALIGN 4 /* multiple of 16 pixels */
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct prp_priv {
+ struct imx_media_dev *md;
+ struct imx_ic_priv *ic_priv;
+ struct media_pad pad[PRP_NUM_PADS];
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* IPU units we require */
+ struct ipu_soc *ipu;
+
+ struct v4l2_subdev *src_sd;
+ struct v4l2_subdev *sink_sd_prpenc;
+ struct v4l2_subdev *sink_sd_prpvf;
+
+ /* the CSI id at link validate */
+ int csi_id;
+
+ struct v4l2_mbus_framefmt format_mbus[PRP_NUM_PADS];
+ const struct imx_media_pixfmt *cc[PRP_NUM_PADS];
+ struct v4l2_fract frame_interval;
+
+ bool stream_on; /* streaming is on */
+};
+
+static inline struct prp_priv *sd_to_priv(struct v4l2_subdev *sd)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+
+ return ic_priv->prp_priv;
+}
+
+static int prp_start(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ priv->ipu = priv->md->ipu[ic_priv->ipu_id];
+
+ /* set IC to receive from CSI or VDI depending on source */
+ if (priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC)
+ ipu_set_ic_src_mux(priv->ipu, 0, true);
+ else
+ ipu_set_ic_src_mux(priv->ipu, priv->csi_id, false);
+
+ return 0;
+}
+
+static void prp_stop(struct prp_priv *priv)
+{
+}
+
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
+}
+
+static struct v4l2_mbus_framefmt *
+__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&ic_priv->sd, cfg, pad);
+ else
+ return &priv->format_mbus[pad];
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int prp_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_mbus_framefmt *fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+ if (!fmt) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ sdformat->format = *fmt;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ const struct imx_media_pixfmt *cc = NULL;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= PRP_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ cc = imx_media_find_ipu_format(0, sdformat->format.code, true);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ cc = imx_media_find_ipu_format(0, code, true);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+ /* Output pads mirror input pad */
+ if (sdformat->pad == PRP_SRC_PAD_PRPENC ||
+ sdformat->pad == PRP_SRC_PAD_PRPVF) {
+ infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD,
+ sdformat->which);
+ sdformat->format = *infmt;
+ }
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_s_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad != PRP_SINK_PAD)
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(ic_priv->dev, "link setup %s -> %s", remote->entity->name,
+ local->entity->name);
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ mutex_lock(&priv->lock);
+
+ if (local->flags & MEDIA_PAD_FL_SINK) {
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ if (priv->sink_sd_prpenc && (remote_sd->grp_id &
+ IMX_MEDIA_GRP_ID_VDIC)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is a source pad */
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ switch (local->index) {
+ case PRP_SRC_PAD_PRPENC:
+ if (priv->sink_sd_prpenc) {
+ ret = -EBUSY;
+ goto out;
+ }
+ if (priv->src_sd && (priv->src_sd->grp_id &
+ IMX_MEDIA_GRP_ID_VDIC)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ priv->sink_sd_prpenc = remote_sd;
+ break;
+ case PRP_SRC_PAD_PRPVF:
+ if (priv->sink_sd_prpvf) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->sink_sd_prpvf = remote_sd;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ } else {
+ switch (local->index) {
+ case PRP_SRC_PAD_PRPENC:
+ priv->sink_sd_prpenc = NULL;
+ break;
+ case PRP_SRC_PAD_PRPVF:
+ priv->sink_sd_prpvf = NULL;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_link_validate(struct v4l2_subdev *sd,
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ struct imx_media_subdev *csi;
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+ if (ret)
+ return ret;
+
+ csi = imx_media_find_pipeline_subdev(priv->md, &ic_priv->sd.entity,
+ IMX_MEDIA_GRP_ID_CSI);
+ if (IS_ERR(csi))
+ csi = NULL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC) {
+ /*
+ * the ->PRPENC link cannot be enabled if the source
+ * is the VDIC
+ */
+ if (priv->sink_sd_prpenc)
+ ret = -EINVAL;
+ goto out;
+ } else {
+ /* the source is a CSI */
+ if (!csi) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ switch (csi->sd->grp_id) {
+ case IMX_MEDIA_GRP_ID_CSI0:
+ priv->csi_id = 0;
+ break;
+ case IMX_MEDIA_GRP_ID_CSI1:
+ priv->csi_id = 1;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->prp_priv;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->src_sd || (!priv->sink_sd_prpenc && !priv->sink_sd_prpvf)) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ dev_dbg(ic_priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+
+ if (enable && !priv->stream_on)
+ ret = prp_start(priv);
+ else if (!enable && priv->stream_on)
+ prp_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int prp_registered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ int i, ret;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ for (i = 0; i < PRP_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == PRP_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ /* set a default mbus format */
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ return ret;
+ }
+
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+ return media_entity_pads_init(&sd->entity, PRP_NUM_PADS, priv->pad);
+}
+
+static struct v4l2_subdev_pad_ops prp_pad_ops = {
+ .enum_mbus_code = prp_enum_mbus_code,
+ .get_fmt = prp_get_fmt,
+ .set_fmt = prp_set_fmt,
+ .link_validate = prp_link_validate,
+};
+
+static struct v4l2_subdev_video_ops prp_video_ops = {
+ .s_stream = prp_s_stream,
+ .g_frame_interval = prp_g_frame_interval,
+ .s_frame_interval = prp_s_frame_interval,
+};
+
+static struct media_entity_operations prp_entity_ops = {
+ .link_setup = prp_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_ops prp_subdev_ops = {
+ .video = &prp_video_ops,
+ .pad = &prp_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops prp_internal_ops = {
+ .registered = prp_registered,
+};
+
+static int prp_init(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv;
+
+ priv = devm_kzalloc(ic_priv->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ mutex_init(&priv->lock);
+ ic_priv->prp_priv = priv;
+ priv->ic_priv = ic_priv;
+
+ return 0;
+}
+
+static void prp_remove(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv = ic_priv->prp_priv;
+
+ mutex_destroy(&priv->lock);
+}
+
+struct imx_ic_ops imx_ic_prp_ops = {
+ .subdev_ops = &prp_subdev_ops,
+ .internal_ops = &prp_internal_ops,
+ .entity_ops = &prp_entity_ops,
+ .init = prp_init,
+ .remove = prp_remove,
+};
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
new file mode 100644
index 0000000..bd89e9b
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -0,0 +1,1219 @@
+/*
+ * V4L2 Capture IC Preprocess Subdev for Freescale i.MX5/6 SOC
+ *
+ * This subdevice handles capture of video frames from the CSI or VDIC,
+ * which are routed directly to the Image Converter preprocess tasks,
+ * for resizing, colorspace conversion, and rotation.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+#include "imx-media.h"
+#include "imx-ic.h"
+
+/*
+ * Min/Max supported width and heights.
+ *
+ * We allow planar output, so we have to align width at the source pad
+ * by 16 pixels to meet IDMAC alignment requirements for possible planar
+ * output.
+ *
+ * TODO: move this into pad format negotiation, if capture device
+ * has not requested a planar format, we should allow 8 pixel
+ * alignment at the source pad.
+ */
+#define MIN_W_SINK 176
+#define MIN_H_SINK 144
+#define MAX_W_SINK 4096
+#define MAX_H_SINK 4096
+#define W_ALIGN_SINK 3 /* multiple of 8 pixels */
+#define H_ALIGN_SINK 1 /* multiple of 2 lines */
+
+#define MAX_W_SRC 1024
+#define MAX_H_SRC 1024
+#define W_ALIGN_SRC 4 /* multiple of 16 pixels */
+#define H_ALIGN_SRC 1 /* multiple of 2 lines */
+
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct prp_priv {
+ struct imx_media_dev *md;
+ struct imx_ic_priv *ic_priv;
+ struct media_pad pad[PRPENCVF_NUM_PADS];
+ /* the video device at output pad */
+ struct imx_media_video_dev *vdev;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* IPU units we require */
+ struct ipu_soc *ipu;
+ struct ipu_ic *ic;
+ struct ipuv3_channel *out_ch;
+ struct ipuv3_channel *rot_in_ch;
+ struct ipuv3_channel *rot_out_ch;
+
+ /* active vb2 buffers to send to video dev sink */
+ struct imx_media_buffer *active_vb2_buf[2];
+ struct imx_media_dma_buf underrun_buf;
+
+ int ipu_buf_num; /* ipu double buffer index: 0-1 */
+
+ /* the sink for the captured frames */
+ struct media_entity *sink;
+ /* the source subdev */
+ struct v4l2_subdev *src_sd;
+
+ /* the attached CSI at stream on */
+ struct v4l2_subdev *csi_sd;
+
+ struct v4l2_mbus_framefmt format_mbus[PRPENCVF_NUM_PADS];
+ const struct imx_media_pixfmt *cc[PRPENCVF_NUM_PADS];
+ struct v4l2_fract frame_interval;
+
+ struct imx_media_dma_buf rot_buf[2];
+
+ /* controls */
+ struct v4l2_ctrl_handler ctrl_hdlr;
+ int rotation; /* degrees */
+ bool hflip;
+ bool vflip;
+
+ /* derived from rotation, hflip, vflip controls */
+ enum ipu_rotate_mode rot_mode;
+
+ spinlock_t irqlock; /* protect eof_irq handler */
+
+ struct timer_list eof_timeout_timer;
+ int eof_irq;
+ int nfb4eof_irq;
+
+ bool stream_on; /* streaming is on */
+ bool last_eof; /* waiting for last EOF at stream off */
+ struct completion last_eof_comp;
+};
+
+static const struct prp_channels {
+ u32 out_ch;
+ u32 rot_in_ch;
+ u32 rot_out_ch;
+} prp_channel[] = {
+ [IC_TASK_ENCODER] = {
+ .out_ch = IPUV3_CHANNEL_IC_PRP_ENC_MEM,
+ .rot_in_ch = IPUV3_CHANNEL_MEM_ROT_ENC,
+ .rot_out_ch = IPUV3_CHANNEL_ROT_ENC_MEM,
+ },
+ [IC_TASK_VIEWFINDER] = {
+ .out_ch = IPUV3_CHANNEL_IC_PRP_VF_MEM,
+ .rot_in_ch = IPUV3_CHANNEL_MEM_ROT_VF,
+ .rot_out_ch = IPUV3_CHANNEL_ROT_VF_MEM,
+ },
+};
+
+static inline struct prp_priv *sd_to_priv(struct v4l2_subdev *sd)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+
+ return ic_priv->task_priv;
+}
+
+static void prp_put_ipu_resources(struct prp_priv *priv)
+{
+ if (!IS_ERR_OR_NULL(priv->ic))
+ ipu_ic_put(priv->ic);
+ priv->ic = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->out_ch))
+ ipu_idmac_put(priv->out_ch);
+ priv->out_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->rot_in_ch))
+ ipu_idmac_put(priv->rot_in_ch);
+ priv->rot_in_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->rot_out_ch))
+ ipu_idmac_put(priv->rot_out_ch);
+ priv->rot_out_ch = NULL;
+}
+
+static int prp_get_ipu_resources(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ int ret, task = ic_priv->task_id;
+
+ priv->ipu = priv->md->ipu[ic_priv->ipu_id];
+
+ priv->ic = ipu_ic_get(priv->ipu, task);
+ if (IS_ERR(priv->ic)) {
+ v4l2_err(&ic_priv->sd, "failed to get IC\n");
+ ret = PTR_ERR(priv->ic);
+ goto out;
+ }
+
+ priv->out_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].out_ch);
+ if (IS_ERR(priv->out_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].out_ch);
+ ret = PTR_ERR(priv->out_ch);
+ goto out;
+ }
+
+ priv->rot_in_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].rot_in_ch);
+ if (IS_ERR(priv->rot_in_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].rot_in_ch);
+ ret = PTR_ERR(priv->rot_in_ch);
+ goto out;
+ }
+
+ priv->rot_out_ch = ipu_idmac_get(priv->ipu,
+ prp_channel[task].rot_out_ch);
+ if (IS_ERR(priv->rot_out_ch)) {
+ v4l2_err(&ic_priv->sd, "could not get IDMAC channel %u\n",
+ prp_channel[task].rot_out_ch);
+ ret = PTR_ERR(priv->rot_out_ch);
+ goto out;
+ }
+
+ return 0;
+out:
+ prp_put_ipu_resources(priv);
+ return ret;
+}
+
+static void prp_vb2_buf_done(struct prp_priv *priv, struct ipuv3_channel *ch)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *done, *next;
+ struct vb2_buffer *vb;
+ dma_addr_t phys;
+
+ done = priv->active_vb2_buf[priv->ipu_buf_num];
+ if (done) {
+ vb = &done->vbuf.vb2_buf;
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+ }
+
+ /* get next queued buffer */
+ next = imx_media_capture_device_next_buf(vdev);
+ if (next) {
+ phys = vb2_dma_contig_plane_dma_addr(&next->vbuf.vb2_buf, 0);
+ priv->active_vb2_buf[priv->ipu_buf_num] = next;
+ } else {
+ phys = priv->underrun_buf.phys;
+ priv->active_vb2_buf[priv->ipu_buf_num] = NULL;
+ }
+
+ if (ipu_idmac_buffer_is_ready(ch, priv->ipu_buf_num))
+ ipu_idmac_clear_buffer(ch, priv->ipu_buf_num);
+
+ ipu_cpmem_set_buffer(ch, priv->ipu_buf_num, phys);
+}
+
+static irqreturn_t prp_eof_interrupt(int irq, void *dev_id)
+{
+ struct prp_priv *priv = dev_id;
+ struct ipuv3_channel *channel;
+
+ spin_lock(&priv->irqlock);
+
+ if (priv->last_eof) {
+ complete(&priv->last_eof_comp);
+ priv->last_eof = false;
+ goto unlock;
+ }
+
+ /*
+ * if there is a CSI in the pipeline, inform it of this EOF
+ * so it can monitor frame intervals.
+ */
+ if (priv->csi_sd)
+ v4l2_subdev_call(priv->csi_sd, core,
+ interrupt_service_routine,
+ 0, NULL);
+
+ channel = (ipu_rot_mode_is_irt(priv->rot_mode)) ?
+ priv->rot_out_ch : priv->out_ch;
+
+ prp_vb2_buf_done(priv, channel);
+
+ /* select new IPU buf */
+ ipu_idmac_select_buffer(channel, priv->ipu_buf_num);
+ /* toggle IPU double-buffer index */
+ priv->ipu_buf_num ^= 1;
+
+ /* bump the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+unlock:
+ spin_unlock(&priv->irqlock);
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t prp_nfb4eof_interrupt(int irq, void *dev_id)
+{
+ struct prp_priv *priv = dev_id;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ static const struct v4l2_event ev = {
+ .type = V4L2_EVENT_NEW_FRAME_BEFORE_EOF,
+ };
+
+ v4l2_err(&ic_priv->sd, "NFB4EOF\n");
+
+ v4l2_subdev_notify_event(&ic_priv->sd, &ev);
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * EOF timeout timer function.
+ */
+/*
+ * EOF timeout timer function. This is an unrecoverable condition
+ * without a stream restart.
+ */
+static void prp_eof_timeout(unsigned long data)
+{
+ struct prp_priv *priv = (struct prp_priv *)data;
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ v4l2_err(&ic_priv->sd, "EOF timeout\n");
+
+ /* signal a fatal error to capture device */
+ imx_media_capture_device_error(vdev);
+}
+
+static void prp_setup_vb2_buf(struct prp_priv *priv, dma_addr_t *phys)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *buf;
+ int i;
+
+ for (i = 0; i < 2; i++) {
+ buf = imx_media_capture_device_next_buf(vdev);
+ priv->active_vb2_buf[i] = buf;
+ phys[i] = vb2_dma_contig_plane_dma_addr(&buf->vbuf.vb2_buf, 0);
+ }
+}
+
+static void prp_unsetup_vb2_buf(struct prp_priv *priv,
+ enum vb2_buffer_state return_state)
+{
+ struct imx_media_buffer *buf;
+ int i;
+
+ /* return any remaining active frames with error */
+ for (i = 0; i < 2; i++) {
+ buf = priv->active_vb2_buf[i];
+ if (buf) {
+ struct vb2_buffer *vb = &buf->vbuf.vb2_buf;
+
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, return_state);
+ }
+ }
+}
+
+static int prp_setup_channel(struct prp_priv *priv,
+ struct ipuv3_channel *channel,
+ enum ipu_rotate_mode rot_mode,
+ dma_addr_t addr0, dma_addr_t addr1,
+ bool rot_swap_width_height)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ const struct imx_media_pixfmt *outcc;
+ struct v4l2_mbus_framefmt *infmt;
+ unsigned int burst_size;
+ struct ipu_image image;
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ipu_cpmem_zero(channel);
+
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ image.rect.width = image.pix.width;
+ image.rect.height = image.pix.height;
+
+ if (rot_swap_width_height) {
+ swap(image.pix.width, image.pix.height);
+ swap(image.rect.width, image.rect.height);
+ /* recalc stride using swapped width */
+ image.pix.bytesperline = outcc->planar ?
+ image.pix.width :
+ (image.pix.width * outcc->bpp) >> 3;
+ }
+
+ image.phys0 = addr0;
+ image.phys1 = addr1;
+
+ ret = ipu_cpmem_set_image(channel, &image);
+ if (ret)
+ return ret;
+
+ if (channel == priv->rot_in_ch ||
+ channel == priv->rot_out_ch) {
+ burst_size = 8;
+ ipu_cpmem_set_block_mode(channel);
+ } else {
+ burst_size = (image.pix.width & 0xf) ? 8 : 16;
+ }
+
+ ipu_cpmem_set_burstsize(channel, burst_size);
+
+ if (rot_mode)
+ ipu_cpmem_set_rotation(channel, rot_mode);
+
+ if (image.pix.field == V4L2_FIELD_NONE &&
+ V4L2_FIELD_HAS_BOTH(infmt->field) &&
+ channel == priv->out_ch)
+ ipu_cpmem_interlaced_scan(channel, image.pix.bytesperline);
+
+ ret = ipu_ic_task_idma_init(priv->ic, channel,
+ image.pix.width, image.pix.height,
+ burst_size, rot_mode);
+ if (ret)
+ return ret;
+
+ ipu_cpmem_set_axi_id(channel, 1);
+
+ ipu_idmac_set_double_buffer(channel, true);
+
+ return 0;
+}
+
+static int prp_setup_rotation(struct prp_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ const struct imx_media_pixfmt *outcc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct v4l2_pix_format *outfmt;
+ dma_addr_t phys[2];
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outfmt = &vdev->fmt.fmt.pix;
+ incc = priv->cc[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->rot_buf[0],
+ outfmt->sizeimage);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "failed to alloc rot_buf[0], %d\n", ret);
+ return ret;
+ }
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->rot_buf[1],
+ outfmt->sizeimage);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "failed to alloc rot_buf[1], %d\n", ret);
+ goto free_rot0;
+ }
+
+ ret = ipu_ic_task_init(priv->ic,
+ infmt->width, infmt->height,
+ outfmt->height, outfmt->width,
+ incc->cs, outcc->cs);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "ipu_ic_task_init failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* init the IC-PRP-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->out_ch, IPU_ROTATE_NONE,
+ priv->rot_buf[0].phys, priv->rot_buf[1].phys,
+ true);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(out_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* init the MEM-->IC-PRP ROT IDMAC channel */
+ ret = prp_setup_channel(priv, priv->rot_in_ch, priv->rot_mode,
+ priv->rot_buf[0].phys, priv->rot_buf[1].phys,
+ true);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(rot_in_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ prp_setup_vb2_buf(priv, phys);
+
+ /* init the destination IC-PRP ROT-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->rot_out_ch, IPU_ROTATE_NONE,
+ phys[0], phys[1],
+ false);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(rot_out_ch) failed, %d\n", ret);
+ goto free_rot1;
+ }
+
+ /* now link IC-PRP-->MEM to MEM-->IC-PRP ROT */
+ ipu_idmac_link(priv->out_ch, priv->rot_in_ch);
+
+ /* enable the IC */
+ ipu_ic_enable(priv->ic);
+
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->out_ch, 0);
+ ipu_idmac_select_buffer(priv->out_ch, 1);
+ ipu_idmac_select_buffer(priv->rot_out_ch, 0);
+ ipu_idmac_select_buffer(priv->rot_out_ch, 1);
+
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->out_ch);
+ ipu_idmac_enable_channel(priv->rot_in_ch);
+ ipu_idmac_enable_channel(priv->rot_out_ch);
+
+ /* and finally enable the IC PRP task */
+ ipu_ic_task_enable(priv->ic);
+
+ return 0;
+
+free_rot1:
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[1]);
+free_rot0:
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[0]);
+ return ret;
+}
+
+static void prp_unsetup_rotation(struct prp_priv *priv)
+{
+ ipu_ic_task_disable(priv->ic);
+
+ ipu_idmac_disable_channel(priv->out_ch);
+ ipu_idmac_disable_channel(priv->rot_in_ch);
+ ipu_idmac_disable_channel(priv->rot_out_ch);
+
+ ipu_idmac_unlink(priv->out_ch, priv->rot_in_ch);
+
+ ipu_ic_disable(priv->ic);
+
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[0]);
+ imx_media_free_dma_buf(priv->md, &priv->rot_buf[1]);
+}
+
+static int prp_setup_norotation(struct prp_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ const struct imx_media_pixfmt *outcc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct v4l2_pix_format *outfmt;
+ dma_addr_t phys[2];
+ int ret;
+
+ infmt = &priv->format_mbus[PRPENCVF_SINK_PAD];
+ outfmt = &vdev->fmt.fmt.pix;
+ incc = priv->cc[PRPENCVF_SINK_PAD];
+ outcc = vdev->cc;
+
+ ret = ipu_ic_task_init(priv->ic,
+ infmt->width, infmt->height,
+ outfmt->width, outfmt->height,
+ incc->cs, outcc->cs);
+ if (ret) {
+ v4l2_err(&ic_priv->sd, "ipu_ic_task_init failed, %d\n", ret);
+ return ret;
+ }
+
+ prp_setup_vb2_buf(priv, phys);
+
+ /* init the IC PRP-->MEM IDMAC channel */
+ ret = prp_setup_channel(priv, priv->out_ch, priv->rot_mode,
+ phys[0], phys[1], false);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "prp_setup_channel(out_ch) failed, %d\n", ret);
+ return ret;
+ }
+
+ ipu_cpmem_dump(priv->out_ch);
+ ipu_ic_dump(priv->ic);
+ ipu_dump(priv->ipu);
+
+ ipu_ic_enable(priv->ic);
+
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->out_ch, 0);
+ ipu_idmac_select_buffer(priv->out_ch, 1);
+
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->out_ch);
+
+ /* enable the IC task */
+ ipu_ic_task_enable(priv->ic);
+
+ return 0;
+}
+
+static void prp_unsetup_norotation(struct prp_priv *priv)
+{
+ ipu_ic_task_disable(priv->ic);
+ ipu_idmac_disable_channel(priv->out_ch);
+ ipu_ic_disable(priv->ic);
+}
+
+static void prp_unsetup(struct prp_priv *priv,
+ enum vb2_buffer_state state)
+{
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ prp_unsetup_rotation(priv);
+ else
+ prp_unsetup_norotation(priv);
+
+ prp_unsetup_vb2_buf(priv, state);
+}
+
+static int prp_start(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_pix_format *outfmt;
+ int ret;
+
+ ret = prp_get_ipu_resources(priv);
+ if (ret)
+ return ret;
+
+ outfmt = &vdev->fmt.fmt.pix;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->underrun_buf,
+ outfmt->sizeimage);
+ if (ret)
+ goto out_put_ipu;
+
+ priv->ipu_buf_num = 0;
+
+ /* init EOF completion waitq */
+ init_completion(&priv->last_eof_comp);
+ priv->last_eof = false;
+
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ ret = prp_setup_rotation(priv);
+ else
+ ret = prp_setup_norotation(priv);
+ if (ret)
+ goto out_free_underrun;
+
+ priv->nfb4eof_irq = ipu_idmac_channel_irq(priv->ipu,
+ priv->out_ch,
+ IPU_IRQ_NFB4EOF);
+ ret = devm_request_irq(ic_priv->dev, priv->nfb4eof_irq,
+ prp_nfb4eof_interrupt, 0,
+ "imx-ic-prp-nfb4eof", priv);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "Error registering NFB4EOF irq: %d\n", ret);
+ goto out_unsetup;
+ }
+
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ priv->eof_irq = ipu_idmac_channel_irq(
+ priv->ipu, priv->rot_out_ch, IPU_IRQ_EOF);
+ else
+ priv->eof_irq = ipu_idmac_channel_irq(
+ priv->ipu, priv->out_ch, IPU_IRQ_EOF);
+
+ ret = devm_request_irq(ic_priv->dev, priv->eof_irq,
+ prp_eof_interrupt, 0,
+ "imx-ic-prp-eof", priv);
+ if (ret) {
+ v4l2_err(&ic_priv->sd,
+ "Error registering eof irq: %d\n", ret);
+ goto out_free_nfb4eof_irq;
+ }
+
+ /* start the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+ return 0;
+
+out_free_nfb4eof_irq:
+ devm_free_irq(ic_priv->dev, priv->nfb4eof_irq, priv);
+out_unsetup:
+ prp_unsetup(priv, VB2_BUF_STATE_QUEUED);
+out_free_underrun:
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+out_put_ipu:
+ prp_put_ipu_resources(priv);
+ return ret;
+}
+
+static void prp_stop(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ unsigned long flags;
+ int ret;
+
+ /* mark next EOF interrupt as the last before stream off */
+ spin_lock_irqsave(&priv->irqlock, flags);
+ priv->last_eof = true;
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+
+ /*
+ * and then wait for interrupt handler to mark completion.
+ */
+ ret = wait_for_completion_timeout(
+ &priv->last_eof_comp,
+ msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+ if (ret == 0)
+ v4l2_warn(&ic_priv->sd, "wait last EOF timeout\n");
+
+ devm_free_irq(ic_priv->dev, priv->eof_irq, priv);
+ devm_free_irq(ic_priv->dev, priv->nfb4eof_irq, priv);
+
+ prp_unsetup(priv, VB2_BUF_STATE_ERROR);
+
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+
+ /* cancel the EOF timeout timer */
+ del_timer_sync(&priv->eof_timeout_timer);
+
+ prp_put_ipu_resources(priv);
+}
+
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ if (code->pad == PRPENCVF_SRC_PAD)
+ return imx_media_enum_format(NULL, &code->code, code->index,
+ true, false);
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
+}
+
+static struct v4l2_mbus_framefmt *
+__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&ic_priv->sd, cfg, pad);
+ else
+ return &priv->format_mbus[pad];
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int prp_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_mbus_framefmt *fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+ if (!fmt) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ sdformat->format = *fmt;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ const struct imx_media_pixfmt *cc;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= PRPENCVF_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (sdformat->pad == PRPENCVF_SRC_PAD) {
+ infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
+ sdformat->which);
+
+ cc = imx_media_find_format(0, sdformat->format.code,
+ true, false);
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0, true, false);
+ cc = imx_media_find_format(0, code, true, false);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ if (sdformat->format.field != V4L2_FIELD_NONE)
+ sdformat->format.field = infmt->field;
+
+ /* IC resizer cannot downsize more than 4:1 */
+ if (ipu_rot_mode_is_irt(priv->rot_mode))
+ v4l_bound_align_image(&sdformat->format.width,
+ infmt->height / 4, MAX_H_SRC,
+ H_ALIGN_SRC,
+ &sdformat->format.height,
+ infmt->width / 4, MAX_W_SRC,
+ W_ALIGN_SRC, S_ALIGN);
+ else
+ v4l_bound_align_image(&sdformat->format.width,
+ infmt->width / 4, MAX_W_SRC,
+ W_ALIGN_SRC,
+ &sdformat->format.height,
+ infmt->height / 4, MAX_H_SRC,
+ H_ALIGN_SRC, S_ALIGN);
+ } else {
+ cc = imx_media_find_ipu_format(0, sdformat->format.code,
+ true);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ cc = imx_media_find_ipu_format(0, code, true);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ v4l_bound_align_image(&sdformat->format.width,
+ MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
+ &sdformat->format.height,
+ MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
+ S_ALIGN);
+ }
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_s_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ mutex_lock(&priv->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad == PRPENCVF_SRC_PAD)
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int prp_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(ic_priv->dev, "link setup %s -> %s", remote->entity->name,
+ local->entity->name);
+
+ mutex_lock(&priv->lock);
+
+ if (local->flags & MEDIA_PAD_FL_SINK) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is the source pad */
+
+ /* the remote must be the device node */
+ if (!is_media_entity_v4l2_video_device(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->sink) {
+ ret = -EBUSY;
+ goto out;
+ }
+ } else {
+ priv->sink = NULL;
+ goto out;
+ }
+
+ priv->sink = remote->entity;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_link_validate(struct v4l2_subdev *sd,
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ struct imx_media_subdev *csi;
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+ if (ret)
+ return ret;
+
+ csi = imx_media_find_pipeline_subdev(priv->md, &ic_priv->sd.entity,
+ IMX_MEDIA_GRP_ID_CSI);
+ if (IS_ERR(csi))
+ csi = NULL;
+
+ mutex_lock(&priv->lock);
+ priv->csi_sd = csi ? csi->sd : NULL;
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct prp_priv *priv = container_of(ctrl->handler,
+ struct prp_priv, ctrl_hdlr);
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ enum ipu_rotate_mode rot_mode;
+ int rotation, ret = 0;
+ bool hflip, vflip;
+
+ mutex_lock(&priv->lock);
+
+ rotation = priv->rotation;
+ hflip = priv->hflip;
+ vflip = priv->vflip;
+
+ switch (ctrl->id) {
+ case V4L2_CID_HFLIP:
+ hflip = (ctrl->val == 1);
+ break;
+ case V4L2_CID_VFLIP:
+ vflip = (ctrl->val == 1);
+ break;
+ case V4L2_CID_ROTATE:
+ rotation = ctrl->val;
+ break;
+ default:
+ v4l2_err(&ic_priv->sd, "Invalid control\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = ipu_degrees_to_rot_mode(&rot_mode, rotation, hflip, vflip);
+ if (ret)
+ goto out;
+
+ if (rot_mode != priv->rot_mode) {
+ goto out;
+
+ /* can't change rotation mid-streaming */
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->rot_mode = rot_mode;
+ priv->rotation = rotation;
+ priv->hflip = hflip;
+ priv->vflip = vflip;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops prp_ctrl_ops = {
+ .s_ctrl = prp_s_ctrl,
+};
+
+static int prp_init_controls(struct prp_priv *priv)
+{
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
+ struct v4l2_ctrl_handler *hdlr = &priv->ctrl_hdlr;
+ int ret;
+
+ v4l2_ctrl_handler_init(hdlr, 3);
+
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_HFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_VFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(hdlr, &prp_ctrl_ops, V4L2_CID_ROTATE,
+ 0, 270, 90, 0);
+
+ ic_priv->sd.ctrl_handler = hdlr;
+
+ if (hdlr->error) {
+ ret = hdlr->error;
+ goto out_free;
+ }
+
+ v4l2_ctrl_handler_setup(hdlr);
+ return 0;
+
+out_free:
+ v4l2_ctrl_handler_free(hdlr);
+ return ret;
+}
+
+static int prp_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imx_ic_priv *ic_priv = v4l2_get_subdevdata(sd);
+ struct prp_priv *priv = ic_priv->task_priv;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+ if (!priv->src_sd || !priv->sink) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ dev_dbg(ic_priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+
+ if (enable && !priv->stream_on)
+ ret = prp_start(priv);
+ else if (!enable && priv->stream_on)
+ prp_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int prp_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ if (sub->type != V4L2_EVENT_NEW_FRAME_BEFORE_EOF)
+ return -EINVAL;
+ if (sub->id != 0)
+ return -EINVAL;
+
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static int prp_unsubscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_unsubscribe(fh, sub);
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int prp_registered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ int i, ret;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
+ if (i == PRPENCVF_SINK_PAD) {
+ priv->pad[i].flags = MEDIA_PAD_FL_SINK;
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ } else {
+ priv->pad[i].flags = MEDIA_PAD_FL_SOURCE;
+ code = 0;
+ }
+
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ return ret;
+ }
+
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+ ret = media_entity_pads_init(&sd->entity, PRPENCVF_NUM_PADS,
+ priv->pad);
+ if (ret)
+ return ret;
+
+ ret = imx_media_capture_device_register(priv->vdev);
+ if (ret)
+ return ret;
+
+ ret = prp_init_controls(priv);
+ if (ret)
+ imx_media_capture_device_unregister(priv->vdev);
+
+ return ret;
+}
+
+static void prp_unregistered(struct v4l2_subdev *sd)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+
+ imx_media_capture_device_unregister(priv->vdev);
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+}
+
+static struct v4l2_subdev_pad_ops prp_pad_ops = {
+ .enum_mbus_code = prp_enum_mbus_code,
+ .get_fmt = prp_get_fmt,
+ .set_fmt = prp_set_fmt,
+ .link_validate = prp_link_validate,
+};
+
+static const struct v4l2_subdev_core_ops prp_core_ops = {
+ .subscribe_event = prp_subscribe_event,
+ .unsubscribe_event = prp_unsubscribe_event,
+};
+
+static struct v4l2_subdev_video_ops prp_video_ops = {
+ .s_stream = prp_s_stream,
+ .g_frame_interval = prp_g_frame_interval,
+ .s_frame_interval = prp_s_frame_interval,
+};
+
+static struct media_entity_operations prp_entity_ops = {
+ .link_setup = prp_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_ops prp_subdev_ops = {
+ .core = &prp_core_ops,
+ .video = &prp_video_ops,
+ .pad = &prp_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops prp_internal_ops = {
+ .registered = prp_registered,
+ .unregistered = prp_unregistered,
+};
+
+static int prp_init(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv;
+
+ priv = devm_kzalloc(ic_priv->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ ic_priv->task_priv = priv;
+ priv->ic_priv = ic_priv;
+
+ spin_lock_init(&priv->irqlock);
+ init_timer(&priv->eof_timeout_timer);
+ priv->eof_timeout_timer.data = (unsigned long)priv;
+ priv->eof_timeout_timer.function = prp_eof_timeout;
+
+ priv->vdev = imx_media_capture_device_init(&ic_priv->sd,
+ PRPENCVF_SRC_PAD);
+ if (IS_ERR(priv->vdev))
+ return PTR_ERR(priv->vdev);
+
+ mutex_init(&priv->lock);
+
+ return 0;
+}
+
+static void prp_remove(struct imx_ic_priv *ic_priv)
+{
+ struct prp_priv *priv = ic_priv->task_priv;
+
+ mutex_destroy(&priv->lock);
+ imx_media_capture_device_remove(priv->vdev);
+}
+
+struct imx_ic_ops imx_ic_prpencvf_ops = {
+ .subdev_ops = &prp_subdev_ops,
+ .internal_ops = &prp_internal_ops,
+ .entity_ops = &prp_entity_ops,
+ .init = prp_init,
+ .remove = prp_remove,
+};
diff --git a/drivers/staging/media/imx/imx-ic.h b/drivers/staging/media/imx/imx-ic.h
new file mode 100644
index 0000000..5535111
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic.h
@@ -0,0 +1,38 @@
+/*
+ * V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#ifndef _IMX_IC_H
+#define _IMX_IC_H
+
+#include <media/v4l2-subdev.h>
+
+struct imx_ic_priv {
+ struct device *dev;
+ struct v4l2_subdev sd;
+ int ipu_id;
+ int task_id;
+ void *prp_priv;
+ void *task_priv;
+};
+
+struct imx_ic_ops {
+ struct v4l2_subdev_ops *subdev_ops;
+ struct v4l2_subdev_internal_ops *internal_ops;
+ struct media_entity_operations *entity_ops;
+
+ int (*init)(struct imx_ic_priv *ic_priv);
+ void (*remove)(struct imx_ic_priv *ic_priv);
+};
+
+extern struct imx_ic_ops imx_ic_prp_ops;
+extern struct imx_ic_ops imx_ic_prpencvf_ops;
+extern struct imx_ic_ops imx_ic_pp_ops;
+
+#endif
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.
Until the OV5652 sensor module compatible with the SabreSD becomes
available for testing, the ov5642 node is currently disabled.
arch/arm/boot/dts/imx6dl-sabresd.dts | 5 ++
arch/arm/boot/dts/imx6q-sabresd.dts | 5 ++
arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 ++++++++++++++++++++++++++++++++-
3 files changed, 123 insertions(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/imx6dl-sabresd.dts b/arch/arm/boot/dts/imx6dl-sabresd.dts
index 1e45f2f..9607afe 100644
--- a/arch/arm/boot/dts/imx6dl-sabresd.dts
+++ b/arch/arm/boot/dts/imx6dl-sabresd.dts
@@ -15,3 +15,8 @@
model = "Freescale i.MX6 DualLite SABRE Smart Device Board";
compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl";
};
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabresd.dts b/arch/arm/boot/dts/imx6q-sabresd.dts
index 9cbdfe7..527772b 100644
--- a/arch/arm/boot/dts/imx6q-sabresd.dts
+++ b/arch/arm/boot/dts/imx6q-sabresd.dts
@@ -23,3 +23,8 @@
&sata {
status = "okay";
};
+
+&ipu1_csi1_from_mipi_vc1 {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
index 63bf95e..643c1d4 100644
--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -10,6 +10,7 @@
* http://www.gnu.org/copyleft/gpl.html
*/
+#include <dt-bindings/clock/imx6qdl-clock.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
@@ -146,6 +147,36 @@
};
};
+&ipu1_csi0_from_ipu1_csi0_mux {
+ bus-width = <8>;
+ data-shift = <12>; /* Lines 19:12 used */
+ hsync-active = <1>;
+ vsync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+ remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
+&mipi_csi {
+ status = "okay";
+
+ port@0 {
+ reg = <0>;
+
+ mipi_csi2_in: endpoint {
+ remote-endpoint = <&ov5640_to_mipi_csi2>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+};
+
&audmux {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_audmux>;
@@ -213,7 +244,32 @@
0x8014 /* 4:FN_DMICCDAT */
0x0000 /* 5:Default */
>;
- };
+ };
+
+ ov5642: camera@3c {
+ compatible = "ovti,ov5642";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5642>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ reg = <0x3c>;
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ AVDD-supply = <&vgen3_reg>; /* 2.8v, rev C board is VGEN3
+ rev B board is VGEN5 */
+ DVDD-supply = <&vgen2_reg>; /* 1.5v*/
+ powerdown-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>;
+ status = "disabled";
+
+ port {
+ ov5642_to_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+ bus-width = <8>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ };
+ };
+ };
};
&i2c2 {
@@ -222,6 +278,32 @@
pinctrl-0 = <&pinctrl_i2c2>;
status = "okay";
+ ov5640: camera@3c {
+ compatible = "ovti,ov5640";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ reg = <0x3c>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ AVDD-supply = <&vgen3_reg>; /* 2.8v, rev C board is VGEN3
+ rev B board is VGEN5 */
+ DVDD-supply = <&vgen2_reg>; /* 1.5v*/
+ powerdown-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>;
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ov5640_to_mipi_csi2: endpoint {
+ remote-endpoint = <&mipi_csi2_in>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
pmic: pfuze100@08 {
compatible = "fsl,pfuze100";
reg = <0x08>;
@@ -425,6 +507,36 @@
>;
};
+ pinctrl_ipu1_csi0: ipu1csi0grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0
+ MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0
+ MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0
+ MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0
+ >;
+ };
+
+ pinctrl_ov5640: ov5640grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT2__GPIO1_IO19 0x1b0b0
+ MX6QDL_PAD_SD1_CLK__GPIO1_IO20 0x1b0b0
+ >;
+ };
+
+ pinctrl_ov5642: ov5642grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+ MX6QDL_PAD_SD1_DAT1__GPIO1_IO17 0x1b0b0
+ >;
+ };
+
pinctrl_pcie: pciegrp {
fsl,pins = <
MX6QDL_PAD_GPIO_17__GPIO7_IO12 0x1b0b0
--
2.7.4
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.
Until the OV5652 sensor module compatible with the SabreSD becomes
available for testing, the ov5642 node is currently disabled.
arch/arm/boot/dts/imx6dl-sabresd.dts | 5 ++
arch/arm/boot/dts/imx6q-sabresd.dts | 5 ++
arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 ++++++++++++++++++++++++++++++++-
3 files changed, 123 insertions(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/imx6dl-sabresd.dts b/arch/arm/boot/dts/imx6dl-sabresd.dts
index 1e45f2f..9607afe 100644
--- a/arch/arm/boot/dts/imx6dl-sabresd.dts
+++ b/arch/arm/boot/dts/imx6dl-sabresd.dts
@@ -15,3 +15,8 @@
model = "Freescale i.MX6 DualLite SABRE Smart Device Board";
compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl";
};
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabresd.dts b/arch/arm/boot/dts/imx6q-sabresd.dts
index 9cbdfe7..527772b 100644
--- a/arch/arm/boot/dts/imx6q-sabresd.dts
+++ b/arch/arm/boot/dts/imx6q-sabresd.dts
@@ -23,3 +23,8 @@
&sata {
status = "okay";
};
+
+&ipu1_csi1_from_mipi_vc1 {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
index 63bf95e..643c1d4 100644
--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -10,6 +10,7 @@
* http://www.gnu.org/copyleft/gpl.html
*/
+#include <dt-bindings/clock/imx6qdl-clock.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
@@ -146,6 +147,36 @@
};
};
+&ipu1_csi0_from_ipu1_csi0_mux {
+ bus-width = <8>;
+ data-shift = <12>; /* Lines 19:12 used */
+ hsync-active = <1>;
+ vsync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+ remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
+&mipi_csi {
+ status = "okay";
+
+ port@0 {
+ reg = <0>;
+
+ mipi_csi2_in: endpoint {
+ remote-endpoint = <&ov5640_to_mipi_csi2>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+};
+
&audmux {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_audmux>;
@@ -213,7 +244,32 @@
0x8014 /* 4:FN_DMICCDAT */
0x0000 /* 5:Default */
>;
- };
+ };
+
+ ov5642: camera@3c {
+ compatible = "ovti,ov5642";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5642>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ reg = <0x3c>;
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ AVDD-supply = <&vgen3_reg>; /* 2.8v, rev C board is VGEN3
+ rev B board is VGEN5 */
+ DVDD-supply = <&vgen2_reg>; /* 1.5v*/
+ powerdown-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>;
+ status = "disabled";
+
+ port {
+ ov5642_to_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+ bus-width = <8>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ };
+ };
+ };
};
&i2c2 {
@@ -222,6 +278,32 @@
pinctrl-0 = <&pinctrl_i2c2>;
status = "okay";
+ ov5640: camera@3c {
+ compatible = "ovti,ov5640";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ reg = <0x3c>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ AVDD-supply = <&vgen3_reg>; /* 2.8v, rev C board is VGEN3
+ rev B board is VGEN5 */
+ DVDD-supply = <&vgen2_reg>; /* 1.5v*/
+ powerdown-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>;
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ov5640_to_mipi_csi2: endpoint {
+ remote-endpoint = <&mipi_csi2_in>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
pmic: pfuze100@08 {
compatible = "fsl,pfuze100";
reg = <0x08>;
@@ -425,6 +507,36 @@
>;
};
+ pinctrl_ipu1_csi0: ipu1csi0grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0
+ MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0
+ MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0
+ MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0
+ >;
+ };
+
+ pinctrl_ov5640: ov5640grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT2__GPIO1_IO19 0x1b0b0
+ MX6QDL_PAD_SD1_CLK__GPIO1_IO20 0x1b0b0
+ >;
+ };
+
+ pinctrl_ov5642: ov5642grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+ MX6QDL_PAD_SD1_DAT1__GPIO1_IO17 0x1b0b0
+ >;
+ };
+
pinctrl_pcie: pciegrp {
fsl,pins = <
MX6QDL_PAD_GPIO_17__GPIO7_IO12 0x1b0b0
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
In version 5:
- ov5640: renamed "pwdn-gpios" to "powerdown-gpios"
- ov5640: add mutex lock around the subdev op entry points.
- ov5640: don't attempt to program the new mode in ov5640_set_fmt().
Instead set a new flag, pending_mode_change, and program the new
mode at s_stream() if flag is set.
- ov5640: implement [gs]_frame_interval. As part of that, create
ov5640_try_frame_interval(), which is used by both [gs]_frame_interval
and [gs]_parm.
- ov5640: don't attempt to set controls in ov5640_s_ctrl(), or at
mode change, do it instead after first power-up.
- video-multiplexer: include link_validate in media_entity_operations.
- video-multiplexer: enforce that output pad frame interval must match
input pad frame interval in vidsw_s_frame_interval().
- video-multiplexer: initialize frame interval to a default 30 fps.
- mipi csi-2: renamed "cfg" clock name property to "ref". This is the
27 MHz mipi csi-2 PLL reference clock.
- mipi csi-2: create a hsfreq_map[] table based on
https://community.nxp.com/docs/DOC-94312. Use it to select
a hsfreqrange_sel value when programming the D-PHY, based on
a max Mbps per lane. This is computed from the source subdev
via V4L2_CID_LINK_FREQ control, and if the subdev doesn't implement
that control, use a default hard-coded max Mbps per lane.
- added required ports property description to imx-media binding doc.
- removed event V4L2_EVENT_FRAME_TIMEOUT. On a frame timeout, which
is always unrecoverable, call vb2_queue_error() instead.
- export the remaining custom events to V4L2_EVENT_FRAME_INTERVAL_ERROR
and V4L2_EVENT_NEW_FRAME_BEFORE_EOF.
- vdic: use V4L2_CID_DEINTERLACING_MODE for motion compensation control
instead of a custom control.
- add v4l2_subdev_link_validate_frame_interval(). Call this in the
link_validate imx-media subdev callbacks and video-multiplexer.
- fix subdev event registration: implementation of subscribe_event()
and unsubscribe_event() subdev ops were missing.
- all calls from the pipeline to the sensor subdev have been removed.
Only the CSI subdev still refers to a sensor, and only to retrieve
its media bus config, which is necessary to setup the CSI interface.
- add mutex locks around the imx-media subdev op entry points.
- completed the propagation of all pad format parameters from sink
pads to source pads within every imx-media subdev.
- implement [gs]_frame_interval in all the imx-media subdevs.
- imx-ic-prpencvf: there isn't necessarily a CSI subdev in the pipeline
in the future, so make sure this is optional when calling the CSI's
FIM.
- the source pads that attach to capture device nodes now require the
IPU internal pixel codes. The capture device translates these to
v4l2 fourcc memory formats.
- fix control inheritance to the capture device. When the pipeline
was modified, the inherited controls were not being refreshed.
v4l2_pipeline_inherit_controls() is now called only in imx-media
link_notify() callback when a pipelink link is disabled or modified.
imx_media_find_pipeline_video_device() is created to locate the
capture device in the pipeline.
- fix a possible race when propagating formats to the capture device.
The subdevs and capture device use different mutex locks when setting
formats. imx_media_capture_device_set_format() is created which acquires
the capture device mutex when updating the capture device format.
- verify all subdevs were bound in the async completion callback.
Philipp Zabel (7):
[media] dt-bindings: Add bindings for video-multiplexer device
ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
connections
add mux and video interface bridge entity functions
platform: add video-multiplexer subdevice driver
media: imx: csi: fix crop rectangle changes in set_fmt
media: imx: csi: add frame skipping support
media: imx: csi: fix crop rectangle reset in sink set_fmt
Russell King (4):
media: imx: add support for bayer formats
media: imx: csi: add support for bayer formats
media: imx: mipi-csi2: enable setting and getting of frame rates
media: imx: csi/fim: add support for frame intervals
Steve Longerbeam (28):
[media] dt-bindings: Add bindings for i.MX media driver
[media] dt/bindings: Add bindings for OV5640
ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
ARM: dts: imx6qdl: add capture-subsystem device
ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabreauto: create i2cmux for i2c3
ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
[media] v4l2: add a frame interval error event
[media] v4l2: add a new-frame before end-of-frame event
[media] v4l2-mc: add a function to inherit controls from a pipeline
[media] v4l: subdev: Add function to validate frame interval
[media] add Omnivision OV5640 sensor driver
UAPI: Add media UAPI Kbuild file
media: Add userspace header file for i.MX
media: Add i.MX media core driver
media: imx: Add Capture Device Interface
media: imx: Add CSI subdev driver
media: imx: Add VDIC subdev driver
media: imx: Add IC subdev drivers
media: imx: Add MIPI CSI-2 Receiver subdev driver
ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
media: imx: csi: add __csi_get_fmt
media: imx: redo pixel format enumeration and negotiation
media: imx: propagate sink pad formats to source pads
.../devicetree/bindings/media/i2c/ov5640.txt | 45 +
Documentation/devicetree/bindings/media/imx.txt | 74 +
.../bindings/media/video-multiplexer.txt | 59 +
Documentation/media/uapi/mediactl/media-types.rst | 22 +
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 12 +
Documentation/media/v4l-drivers/imx.rst | 560 +++++
Documentation/media/videodev2.h.rst.exceptions | 2 +
arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 +
arch/arm/boot/dts/imx6dl-sabresd.dts | 5 +
arch/arm/boot/dts/imx6dl.dtsi | 185 ++
arch/arm/boot/dts/imx6q-sabrelite.dts | 5 +
arch/arm/boot/dts/imx6q-sabresd.dts | 5 +
arch/arm/boot/dts/imx6q.dtsi | 121 ++
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 144 +-
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +-
arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +-
arch/arm/boot/dts/imx6qdl.dtsi | 17 +-
arch/arm/configs/imx_v6_v7_defconfig | 11 +
drivers/media/i2c/Kconfig | 7 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++
drivers/media/platform/Kconfig | 8 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/video-multiplexer.c | 498 +++++
drivers/media/v4l2-core/v4l2-mc.c | 48 +
drivers/media/v4l2-core/v4l2-subdev.c | 50 +
drivers/staging/media/Kconfig | 2 +
drivers/staging/media/Makefile | 1 +
drivers/staging/media/imx/Kconfig | 20 +
drivers/staging/media/imx/Makefile | 12 +
drivers/staging/media/imx/TODO | 17 +
drivers/staging/media/imx/imx-ic-common.c | 113 +
drivers/staging/media/imx/imx-ic-prp.c | 497 +++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 1236 +++++++++++
drivers/staging/media/imx/imx-ic.h | 38 +
drivers/staging/media/imx/imx-media-capture.c | 694 ++++++
drivers/staging/media/imx/imx-media-csi.c | 1595 ++++++++++++++
drivers/staging/media/imx/imx-media-dev.c | 522 +++++
drivers/staging/media/imx/imx-media-fim.c | 463 ++++
drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++
drivers/staging/media/imx/imx-media-of.c | 267 +++
drivers/staging/media/imx/imx-media-utils.c | 1009 +++++++++
drivers/staging/media/imx/imx-media-vdic.c | 949 +++++++++
drivers/staging/media/imx/imx-media.h | 311 +++
drivers/staging/media/imx/imx6-mipi-csi2.c | 725 +++++++
include/media/imx.h | 15 +
include/media/v4l2-mc.h | 25 +
include/media/v4l2-subdev.h | 10 +
include/uapi/Kbuild | 1 +
include/uapi/linux/media.h | 6 +
include/uapi/linux/v4l2-controls.h | 4 +
include/uapi/linux/videodev2.h | 2 +
include/uapi/media/Kbuild | 2 +
include/uapi/media/imx.h | 21 +
54 files changed, 13262 insertions(+), 27 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
create mode 100644 Documentation/devicetree/bindings/media/imx.txt
create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
create mode 100644 Documentation/media/v4l-drivers/imx.rst
create mode 100644 drivers/media/i2c/ov5640.c
create mode 100644 drivers/media/platform/video-multiplexer.c
create mode 100644 drivers/staging/media/imx/Kconfig
create mode 100644 drivers/staging/media/imx/Makefile
create mode 100644 drivers/staging/media/imx/TODO
create mode 100644 drivers/staging/media/imx/imx-media-csi.c
create mode 100644 drivers/staging/media/imx/imx-media-dev.c
create mode 100644 drivers/staging/media/imx/imx-media-fim.c
create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c
create mode 100644 drivers/staging/media/imx/imx-media-of.c
create mode 100644 drivers/staging/media/imx/imx-media-utils.c
create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
create mode 100644 drivers/staging/media/imx/imx-media.h
create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
create mode 100644 include/media/imx.h
create mode 100644 include/uapi/media/Kbuild
create mode 100644 include/uapi/media/imx.h
--
2.7.4
- ov5640: renamed "pwdn-gpios" to "powerdown-gpios"
- ov5640: add mutex lock around the subdev op entry points.
- ov5640: don't attempt to program the new mode in ov5640_set_fmt().
Instead set a new flag, pending_mode_change, and program the new
mode at s_stream() if flag is set.
- ov5640: implement [gs]_frame_interval. As part of that, create
ov5640_try_frame_interval(), which is used by both [gs]_frame_interval
and [gs]_parm.
- ov5640: don't attempt to set controls in ov5640_s_ctrl(), or at
mode change, do it instead after first power-up.
- video-multiplexer: include link_validate in media_entity_operations.
- video-multiplexer: enforce that output pad frame interval must match
input pad frame interval in vidsw_s_frame_interval().
- video-multiplexer: initialize frame interval to a default 30 fps.
- mipi csi-2: renamed "cfg" clock name property to "ref". This is the
27 MHz mipi csi-2 PLL reference clock.
- mipi csi-2: create a hsfreq_map[] table based on
https://community.nxp.com/docs/DOC-94312. Use it to select
a hsfreqrange_sel value when programming the D-PHY, based on
a max Mbps per lane. This is computed from the source subdev
via V4L2_CID_LINK_FREQ control, and if the subdev doesn't implement
that control, use a default hard-coded max Mbps per lane.
- added required ports property description to imx-media binding doc.
- removed event V4L2_EVENT_FRAME_TIMEOUT. On a frame timeout, which
is always unrecoverable, call vb2_queue_error() instead.
- export the remaining custom events to V4L2_EVENT_FRAME_INTERVAL_ERROR
and V4L2_EVENT_NEW_FRAME_BEFORE_EOF.
- vdic: use V4L2_CID_DEINTERLACING_MODE for motion compensation control
instead of a custom control.
- add v4l2_subdev_link_validate_frame_interval(). Call this in the
link_validate imx-media subdev callbacks and video-multiplexer.
- fix subdev event registration: implementation of subscribe_event()
and unsubscribe_event() subdev ops were missing.
- all calls from the pipeline to the sensor subdev have been removed.
Only the CSI subdev still refers to a sensor, and only to retrieve
its media bus config, which is necessary to setup the CSI interface.
- add mutex locks around the imx-media subdev op entry points.
- completed the propagation of all pad format parameters from sink
pads to source pads within every imx-media subdev.
- implement [gs]_frame_interval in all the imx-media subdevs.
- imx-ic-prpencvf: there isn't necessarily a CSI subdev in the pipeline
in the future, so make sure this is optional when calling the CSI's
FIM.
- the source pads that attach to capture device nodes now require the
IPU internal pixel codes. The capture device translates these to
v4l2 fourcc memory formats.
- fix control inheritance to the capture device. When the pipeline
was modified, the inherited controls were not being refreshed.
v4l2_pipeline_inherit_controls() is now called only in imx-media
link_notify() callback when a pipelink link is disabled or modified.
imx_media_find_pipeline_video_device() is created to locate the
capture device in the pipeline.
- fix a possible race when propagating formats to the capture device.
The subdevs and capture device use different mutex locks when setting
formats. imx_media_capture_device_set_format() is created which acquires
the capture device mutex when updating the capture device format.
- verify all subdevs were bound in the async completion callback.
Philipp Zabel (7):
[media] dt-bindings: Add bindings for video-multiplexer device
ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
connections
add mux and video interface bridge entity functions
platform: add video-multiplexer subdevice driver
media: imx: csi: fix crop rectangle changes in set_fmt
media: imx: csi: add frame skipping support
media: imx: csi: fix crop rectangle reset in sink set_fmt
Russell King (4):
media: imx: add support for bayer formats
media: imx: csi: add support for bayer formats
media: imx: mipi-csi2: enable setting and getting of frame rates
media: imx: csi/fim: add support for frame intervals
Steve Longerbeam (28):
[media] dt-bindings: Add bindings for i.MX media driver
[media] dt/bindings: Add bindings for OV5640
ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
ARM: dts: imx6qdl: add capture-subsystem device
ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabreauto: create i2cmux for i2c3
ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
[media] v4l2: add a frame interval error event
[media] v4l2: add a new-frame before end-of-frame event
[media] v4l2-mc: add a function to inherit controls from a pipeline
[media] v4l: subdev: Add function to validate frame interval
[media] add Omnivision OV5640 sensor driver
UAPI: Add media UAPI Kbuild file
media: Add userspace header file for i.MX
media: Add i.MX media core driver
media: imx: Add Capture Device Interface
media: imx: Add CSI subdev driver
media: imx: Add VDIC subdev driver
media: imx: Add IC subdev drivers
media: imx: Add MIPI CSI-2 Receiver subdev driver
ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
media: imx: csi: add __csi_get_fmt
media: imx: redo pixel format enumeration and negotiation
media: imx: propagate sink pad formats to source pads
.../devicetree/bindings/media/i2c/ov5640.txt | 45 +
Documentation/devicetree/bindings/media/imx.txt | 74 +
.../bindings/media/video-multiplexer.txt | 59 +
Documentation/media/uapi/mediactl/media-types.rst | 22 +
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 12 +
Documentation/media/v4l-drivers/imx.rst | 560 +++++
Documentation/media/videodev2.h.rst.exceptions | 2 +
arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 +
arch/arm/boot/dts/imx6dl-sabresd.dts | 5 +
arch/arm/boot/dts/imx6dl.dtsi | 185 ++
arch/arm/boot/dts/imx6q-sabrelite.dts | 5 +
arch/arm/boot/dts/imx6q-sabresd.dts | 5 +
arch/arm/boot/dts/imx6q.dtsi | 121 ++
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 144 +-
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +-
arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +-
arch/arm/boot/dts/imx6qdl.dtsi | 17 +-
arch/arm/configs/imx_v6_v7_defconfig | 11 +
drivers/media/i2c/Kconfig | 7 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++
drivers/media/platform/Kconfig | 8 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/video-multiplexer.c | 498 +++++
drivers/media/v4l2-core/v4l2-mc.c | 48 +
drivers/media/v4l2-core/v4l2-subdev.c | 50 +
drivers/staging/media/Kconfig | 2 +
drivers/staging/media/Makefile | 1 +
drivers/staging/media/imx/Kconfig | 20 +
drivers/staging/media/imx/Makefile | 12 +
drivers/staging/media/imx/TODO | 17 +
drivers/staging/media/imx/imx-ic-common.c | 113 +
drivers/staging/media/imx/imx-ic-prp.c | 497 +++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 1236 +++++++++++
drivers/staging/media/imx/imx-ic.h | 38 +
drivers/staging/media/imx/imx-media-capture.c | 694 ++++++
drivers/staging/media/imx/imx-media-csi.c | 1595 ++++++++++++++
drivers/staging/media/imx/imx-media-dev.c | 522 +++++
drivers/staging/media/imx/imx-media-fim.c | 463 ++++
drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++
drivers/staging/media/imx/imx-media-of.c | 267 +++
drivers/staging/media/imx/imx-media-utils.c | 1009 +++++++++
drivers/staging/media/imx/imx-media-vdic.c | 949 +++++++++
drivers/staging/media/imx/imx-media.h | 311 +++
drivers/staging/media/imx/imx6-mipi-csi2.c | 725 +++++++
include/media/imx.h | 15 +
include/media/v4l2-mc.h | 25 +
include/media/v4l2-subdev.h | 10 +
include/uapi/Kbuild | 1 +
include/uapi/linux/media.h | 6 +
include/uapi/linux/v4l2-controls.h | 4 +
include/uapi/linux/videodev2.h | 2 +
include/uapi/media/Kbuild | 2 +
include/uapi/media/imx.h | 21 +
54 files changed, 13262 insertions(+), 27 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
create mode 100644 Documentation/devicetree/bindings/media/imx.txt
create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
create mode 100644 Documentation/media/v4l-drivers/imx.rst
create mode 100644 drivers/media/i2c/ov5640.c
create mode 100644 drivers/media/platform/video-multiplexer.c
create mode 100644 drivers/staging/media/imx/Kconfig
create mode 100644 drivers/staging/media/imx/Makefile
create mode 100644 drivers/staging/media/imx/TODO
create mode 100644 drivers/staging/media/imx/imx-ic-common.c
create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
create mode 100644 drivers/staging/media/imx/imx-ic.h
create mode 100644 drivers/staging/media/imx/imx-media-capture.c
create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
create mode 100644 drivers/staging/media/imx/imx-ic.h
create mode 100644 drivers/staging/media/imx/imx-media-csi.c
create mode 100644 drivers/staging/media/imx/imx-media-dev.c
create mode 100644 drivers/staging/media/imx/imx-media-fim.c
create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c
create mode 100644 drivers/staging/media/imx/imx-media-of.c
create mode 100644 drivers/staging/media/imx/imx-media-utils.c
create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
create mode 100644 drivers/staging/media/imx/imx-media.h
create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
create mode 100644 include/media/imx.h
create mode 100644 include/uapi/media/Kbuild
create mode 100644 include/uapi/media/imx.h
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
Add bindings documentation for the video multiplexer device.
Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
.../bindings/media/video-multiplexer.txt | 59 ++++++++++++++++++++++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
diff --git a/Documentation/devicetree/bindings/media/video-multiplexer.txt b/Documentation/devicetree/bindings/media/video-multiplexer.txt
new file mode 100644
index 0000000..9d133d9
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/video-multiplexer.txt
@@ -0,0 +1,59 @@
+Video Multiplexer
+=================
+
+Video multiplexers allow to select between multiple input ports. Video received
+on the active input port is passed through to the output port. Muxes described
+by this binding may be controlled by a syscon register bitfield or by a GPIO.
+
+Required properties:
+- compatible : should be "video-multiplexer"
+- reg: should be register base of the register containing the control bitfield
+- bit-mask: bitmask of the control bitfield in the control register
+- bit-shift: bit offset of the control bitfield in the control register
+- gpios: alternatively to reg, bit-mask, and bit-shift, a single GPIO phandle
+ may be given to switch between two inputs
+- #address-cells: should be <1>
+- #size-cells: should be <0>
+- port@*: at least three port nodes containing endpoints connecting to the
+ source and sink devices according to of_graph bindings. The last port is
+ the output port, all others are inputs.
+
+Example:
+
+syscon {
+ compatible = "syscon", "simple-mfd";
+
+ mux {
+ compatible = "video-multiplexer";
+ /* Single bit (1 << 19) in syscon register 0x04: */
+ reg = <0x04>;
+ bit-mask = <1>;
+ bit-shift = <19>;
+ remote-endpoint = <&video_source0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ mux_in1: endpoint {
+ remote-endpoint = <&video_source1_out>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ mux_out: endpoint {
+ remote-endpoint = <&capture_interface_in>;
+ };
+ };
+ };
+};
--
2.7.4
Add bindings documentation for the video multiplexer device.
Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
.../bindings/media/video-multiplexer.txt | 59 ++++++++++++++++++++++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
diff --git a/Documentation/devicetree/bindings/media/video-multiplexer.txt b/Documentation/devicetree/bindings/media/video-multiplexer.txt
new file mode 100644
index 0000000..9d133d9
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/video-multiplexer.txt
@@ -0,0 +1,59 @@
+Video Multiplexer
+=================
+
+Video multiplexers allow to select between multiple input ports. Video received
+on the active input port is passed through to the output port. Muxes described
+by this binding may be controlled by a syscon register bitfield or by a GPIO.
+
+Required properties:
+- compatible : should be "video-multiplexer"
+- reg: should be register base of the register containing the control bitfield
+- bit-mask: bitmask of the control bitfield in the control register
+- bit-shift: bit offset of the control bitfield in the control register
+- gpios: alternatively to reg, bit-mask, and bit-shift, a single GPIO phandle
+ may be given to switch between two inputs
+- #address-cells: should be <1>
+- #size-cells: should be <0>
+- port@*: at least three port nodes containing endpoints connecting to the
+ source and sink devices according to of_graph bindings. The last port is
+ the output port, all others are inputs.
+
+Example:
+
+syscon {
+ compatible = "syscon", "simple-mfd";
+
+ mux {
+ compatible = "video-multiplexer";
+ /* Single bit (1 << 19) in syscon register 0x04: */
+ reg = <0x04>;
+ bit-mask = <1>;
+ bit-shift = <19>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ mux_in0: endpoint {
+ reg = <0>;
+
+ remote-endpoint = <&video_source0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ mux_in1: endpoint {
+ remote-endpoint = <&video_source1_out>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ mux_out: endpoint {
+ remote-endpoint = <&capture_interface_in>;
+ };
+ };
+ };
+};
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
Adds MIPI CSI-2 Receiver subdev driver. This subdev is required
for sensors with a MIPI CSI2 interface.
drivers/staging/media/imx/Makefile | 1 +
drivers/staging/media/imx/imx6-mipi-csi2.c | 689 +++++++++++++++++++++++++++++
2 files changed, 690 insertions(+)
create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 878a126..3569625 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
new file mode 100644
index 0000000..1a71b40
--- /dev/null
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -0,0 +1,689 @@
+/*
+ * MIPI CSI-2 Receiver Subdev for Freescale i.MX6 SOC.
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/irq.h>
+#include <media/v4l2-subdev.h>
+#include "imx-media.h"
+
+/*
+ * there must be 5 pads: 1 input pad from sensor, and
+ * the 4 virtual channel output pads
+ */
+#define CSI2_SINK_PAD 0
+#define CSI2_NUM_SINK_PADS 1
+#define CSI2_NUM_SRC_PADS 4
+#define CSI2_NUM_PADS 5
+
+/*
+ * The default maximum bit-rate per lane in Mbps, if the
+ * source subdev does not provide V4L2_CID_LINK_FREQ.
+ */
+#define CSI2_DEFAULT_MAX_MBPS 849
+
+struct csi2_dev {
+ struct clk *dphy_clk;
+ struct clk *pllref_clk;
+ struct clk *pix_clk; /* what is this? */
+ void __iomem *base;
+ struct v4l2_of_bus_mipi_csi2 bus;
+
+ int power_count;
+ bool stream_on;
+ struct v4l2_subdev *src_sd;
+ bool sink_linked[CSI2_NUM_SRC_PADS];
+};
+
+#define DEVICE_NAME "imx6-mipi-csi2"
+
+/* Register offsets */
+#define CSI2_VERSION 0x000
+#define CSI2_N_LANES 0x004
+#define CSI2_PHY_SHUTDOWNZ 0x008
+#define CSI2_DPHY_RSTZ 0x00c
+#define CSI2_RESETN 0x010
+#define CSI2_PHY_STATE 0x014
+#define PHY_STOPSTATEDATA_BIT 4
+#define PHY_STOPSTATEDATA(n) BIT(PHY_STOPSTATEDATA_BIT + (n))
+#define PHY_RXCLKACTIVEHS BIT(8)
+#define PHY_RXULPSCLKNOT BIT(9)
+#define PHY_STOPSTATECLK BIT(10)
+#define CSI2_DATA_IDS_1 0x018
+#define CSI2_DATA_IDS_2 0x01c
+#define CSI2_ERR1 0x020
+#define CSI2_ERR2 0x024
+#define CSI2_MSK1 0x028
+#define CSI2_MSK2 0x02c
+#define CSI2_PHY_TST_CTRL0 0x030
+#define PHY_TESTCLR BIT(0)
+#define PHY_TESTCLK BIT(1)
+#define CSI2_PHY_TST_CTRL1 0x034
+#define PHY_TESTEN BIT(16)
+#define CSI2_SFT_RESET 0xf00
+
+static inline struct csi2_dev *sd_to_dev(struct v4l2_subdev *sdev)
+{
+ return container_of(sdev, struct csi2_dev, sd);
+}
+
+static void csi2_enable(struct csi2_dev *csi2, bool enable)
+{
+ if (enable) {
+ writel(0x1, csi2->base + CSI2_PHY_SHUTDOWNZ);
+ writel(0x1, csi2->base + CSI2_DPHY_RSTZ);
+ writel(0x1, csi2->base + CSI2_RESETN);
+ } else {
+ writel(0x0, csi2->base + CSI2_PHY_SHUTDOWNZ);
+ writel(0x0, csi2->base + CSI2_DPHY_RSTZ);
+ writel(0x0, csi2->base + CSI2_RESETN);
+ }
+}
+
+static void csi2_set_lanes(struct csi2_dev *csi2)
+{
+ int lanes = csi2->bus.num_data_lanes;
+
+ writel(lanes - 1, csi2->base + CSI2_N_LANES);
+}
+
+static void dw_mipi_csi2_phy_write(struct csi2_dev *csi2,
+ u32 test_code, u32 test_data)
+{
+ /* Clear PHY test interface */
+ writel(PHY_TESTCLR, csi2->base + CSI2_PHY_TST_CTRL0);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Raise test interface strobe signal */
+ writel(PHY_TESTCLK, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Configure address write on falling edge and lower strobe signal */
+ writel(PHY_TESTEN | test_code, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Configure data write on rising edge and raise strobe signal */
+ writel(test_data, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(PHY_TESTCLK, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Clear strobe signal */
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+}
+
+/*
+ * This table is based on the table documented at
+ * https://community.nxp.com/docs/DOC-94312. It assumes
+ * a 27MHz D-PHY pll reference clock.
+ */
+static const struct {
+ u32 max_mbps;
+ u32 hsfreqrange_sel;
+} hsfreq_map[] = {
+ { 90, 0x00}, {100, 0x20}, {110, 0x40}, {125, 0x02},
+ {140, 0x22}, {150, 0x42}, {160, 0x04}, {180, 0x24},
+ {200, 0x44}, {210, 0x06}, {240, 0x26}, {250, 0x46},
+ {270, 0x08}, {300, 0x28}, {330, 0x48}, {360, 0x2a},
+ {400, 0x4a}, {450, 0x0c}, {500, 0x2c}, {550, 0x0e},
+ {600, 0x2e}, {650, 0x10}, {700, 0x30}, {750, 0x12},
+ {800, 0x32}, {850, 0x14}, {900, 0x34}, {950, 0x54},
+ {1000, 0x74},
+};
+
+static int max_mbps_to_hsfreqrange_sel(u32 max_mbps)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(hsfreq_map); i++)
+ if (hsfreq_map[i].max_mbps > max_mbps)
+ return hsfreq_map[i].hsfreqrange_sel;
+
+{
+ struct v4l2_ctrl *ctrl;
+ u32 mbps_per_lane;
+ int sel;
+
+ ctrl = v4l2_ctrl_find(csi2->src_sd->ctrl_handler,
+ V4L2_CID_LINK_FREQ);
+ if (!ctrl)
+ mbps_per_lane = CSI2_DEFAULT_MAX_MBPS;
+ else
+ mbps_per_lane = DIV_ROUND_UP_ULL(2 * ctrl->qmenu_int[ctrl->val],
+ USEC_PER_SEC);
+
+ sel = max_mbps_to_hsfreqrange_sel(mbps_per_lane);
+ if (sel < 0)
+ return sel;
+
+ dw_mipi_csi2_phy_write(csi2, 0x44, sel);
+ * Waits for ultra-low-power state on D-PHY clock lane. This is currently
+ * unused and may not be needed at all, but keep around just in case.
+ */
+static int __maybe_unused csi2_dphy_wait_ulp(struct csi2_dev *csi2)
+{
+ u32 reg;
+ int ret;
+
+ /* wait for ULP on clock lane */
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ !(reg & PHY_RXULPSCLKNOT), 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "ULP timeout, phy_state = 0x%08x\n", reg);
+ ret = readl_poll_timeout(csi2->base + CSI2_ERR1, reg,
+ reg == 0x0, 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "stable bus timeout, err1 = 0x%08x\n", reg);
+}
+
+/* Waits for low-power LP-11 state on data and clock lanes. */
+static int csi2_dphy_wait_stopstate(struct csi2_dev *csi2)
+{
+ u32 mask, reg;
+ int ret;
+
+ mask = PHY_STOPSTATECLK |
+ ((csi2->bus.num_data_lanes - 1) << PHY_STOPSTATEDATA_BIT);
+
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ (reg & mask) == mask, 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "LP-11 timeout, phy_state = 0x%08x\n", reg);
+}
+
+/*
+ * Wait for active clock on the clock lane.
+ *
+ * FIXME: Currently unused, but it should be! It should be called
+ * from csi2_s_stream() below, at stream ON, but the required sequence
+ * of MIPI CSI-2 startup does not allow for an opportunity for this to
+ * be called. The sequence as specified in the i.MX6 reference manual
+ * is as follows:
+ *
+ * 1. Deassert presetn signal (global reset).
+ * It's not clear what this "global reset" signal is (maybe APB
+ * global reset), but in any case this step corresponds to
+ * csi2_s_power(ON) here.
+ *
+ * 2. Configure MIPI Camera Sensor to put all Tx lanes in PL-11 state.
+ * This must be carried out by the MIPI sensor's s_power(ON) subdev
+ * op.
+ *
+ * 3. D-PHY initialization.
+ * 4. CSI2 Controller programming (Set N_LANES, deassert PHY_SHUTDOWNZ,
+ * deassert PHY_RSTZ, deassert CSI2_RESETN).
+ * 5. Read the PHY status register (PHY_STATE) to confirm that all data and
+ * clock lanes of the D-PHY are in Stop State.
+ * These steps (3,4,5) are carried out by csi2_s_stream(ON) here.
+ *
+ * 6. Configure the MIPI Camera Sensor to start transmitting a clock on the
+ * D-PHY clock lane.
+ * This must be carried out by the MIPI sensor's s_stream(ON) subdev
+ * op.
+ *
+ * 7. CSI2 Controller programming - Read the PHY status register (PHY_STATE)
+ * to confirm that the D-PHY is receiving a clock on the D-PHY clock lane.
+ * This is implemented by this unused function, and _should_ be called
+ * by csi2_s_stream(ON) here, but csi2_s_stream(ON) has been taken up
+ * by steps 3,4,5 above already.
+ *
+ * In summary, a temporary solution would require a hard-coded delay in the
+ * MIPI sensor's s_stream(ON) op, to allow time for a stable clock lane.
+ *
+ * A longer term solution might be to create a new subdev op, perhaps
+ * called prepare_stream, that can be implemented here, and would be
+ * assigned steps 3,4,5. Then csi2_s_stream(ON) would become available
+ * as step 7.
+ */
+static int __maybe_unused csi2_dphy_wait_clock_lane(struct csi2_dev *csi2)
+{
+ u32 reg;
+ int ret;
+
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ (reg & PHY_RXCLKACTIVEHS), 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "clock lane timeout, phy_state = 0x%08x\n",
+ reg);
+static int csi2_s_power(struct v4l2_subdev *sd, int on)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+
+ /*
+ * If the power count is modified from 0 to != 0 or from != 0 to 0,
+ * update the power state.
+ */
+ if (csi2->power_count == !on) {
+ if (on) {
+ dev_dbg(csi2->dev, "power ON\n");
+ ret = clk_prepare_enable(csi2->pllref_clk);
+ if (ret) {
+ clk_disable_unprepare(csi2->pllref_clk);
+ clk_disable_unprepare(csi2->dphy_clk);
+ clk_disable_unprepare(csi2->pllref_clk);
+ }
+ }
+
+ /* Update the power count. */
+ csi2->power_count += on ? 1 : -1;
+ WARN_ON(csi2->power_count < 0);
+out:
+ mutex_unlock(&csi2->lock);
+static int csi2_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ int i, ret = 0;
+
+ mutex_lock(&csi2->lock);
+
+ if (!csi2->src_sd) {
+ if (csi2->sink_linked[i])
+ break;
+ }
+ if (i >= CSI2_NUM_SRC_PADS) {
+ dev_dbg(csi2->dev, "stream ON\n");
+
+ ret = clk_prepare_enable(csi2->pix_clk);
+ ret = csi2_dphy_init(csi2);
+ if (ret) {
+ clk_disable_unprepare(csi2->pix_clk);
+ csi2_set_lanes(csi2);
+ csi2_enable(csi2, true);
+
+ /* Step 5 */
+ ret = csi2_dphy_wait_stopstate(csi2);
+ if (ret) {
+ csi2_enable(csi2, false);
+ clk_disable_unprepare(csi2->pix_clk);
+ goto out;
+ }
+ } else if (!enable && csi2->stream_on) {
+ dev_dbg(csi2->dev, "stream OFF\n");
+ csi2_enable(csi2, false);
+ clk_disable_unprepare(csi2->pix_clk);
+ }
+
+ csi2->stream_on = enable;
+out:
+ mutex_unlock(&csi2->lock);
+
+ if (local->flags & MEDIA_PAD_FL_SOURCE) {
+ } else {
+ csi2->sink_linked[local->index] = false;
+ }
+ } else {
+ } else {
+ csi2->src_sd = NULL;
+ }
+ }
+
+out:
+ mutex_unlock(&csi2->lock);
+ sdformat->pad);
+ else
+ fmt = &csi2->format_mbus;
+ return -EINVAL;
+
+ mutex_lock(&csi2->lock);
+
+ if (csi2->stream_on) {
+ sdformat->format = csi2->format_mbus;
+out:
+ mutex_unlock(&csi2->lock);
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ csi2->pad[i].flags = (i == CSI2_SINK_PAD) ?
+ 640, 480, 0, V4L2_FIELD_NONE, NULL);
+}
+
+static struct media_entity_operations csi2_entity_ops = {
+ .link_setup = csi2_link_setup,
+ .s_power = csi2_s_power,
+};
+
+static struct v4l2_subdev_video_ops csi2_video_ops = {
+ .s_stream = csi2_s_stream,
+};
+
+static struct v4l2_subdev_pad_ops csi2_pad_ops = {
+ .get_fmt = csi2_get_fmt,
+ .set_fmt = csi2_set_fmt,
+ .link_validate = csi2_link_validate,
+};
+
+static struct v4l2_subdev_ops csi2_subdev_ops = {
+ .core = &csi2_core_ops,
+ .video = &csi2_video_ops,
+ .pad = &csi2_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops csi2_internal_ops = {
+ .registered = csi2_registered,
+};
+
+static int csi2_parse_endpoints(struct csi2_dev *csi2)
+{
+ struct device_node *node = csi2->dev->of_node;
+ struct device_node *epnode;
+ struct v4l2_of_endpoint ep;
+
+ epnode = of_graph_get_endpoint_by_regs(node, 0, -1);
+ if (!epnode) {
+ v4l2_err(&csi2->sd, "failed to get sink endpoint node\n");
+ of_node_put(epnode);
+
+ if (ep.bus_type != V4L2_MBUS_CSI2) {
+ v4l2_err(&csi2->sd, "invalid bus type, must be MIPI CSI2\n");
+
+ dev_dbg(csi2->dev, "data lanes: %d\n", csi2->bus.num_data_lanes);
+ dev_dbg(csi2->dev, "flags: 0x%08x\n", csi2->bus.flags);
+{
+ struct csi2_dev *csi2;
+ struct resource *res;
+ int ret;
+
+ csi2 = devm_kzalloc(&pdev->dev, sizeof(*csi2), GFP_KERNEL);
+ if (!csi2)
+ return -ENOMEM;
+
+ csi2->dev = &pdev->dev;
+
+ v4l2_subdev_init(&csi2->sd, &csi2_subdev_ops);
+ v4l2_set_subdevdata(&csi2->sd, &pdev->dev);
+ csi2->sd.internal_ops = &csi2_internal_ops;
+ csi2->sd.entity.ops = &csi2_entity_ops;
+ csi2->sd.dev = &pdev->dev;
+ csi2->sd.owner = THIS_MODULE;
+ csi2->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ strcpy(csi2->sd.name, DEVICE_NAME);
+ csi2->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+ csi2->sd.grp_id = IMX_MEDIA_GRP_ID_CSI2;
+
+ ret = csi2_parse_endpoints(csi2);
+ if (IS_ERR(csi2->pllref_clk)) {
+ v4l2_err(&csi2->sd, "failed to get pll reference clock\n");
+ ret = PTR_ERR(csi2->pllref_clk);
+ if (IS_ERR(csi2->dphy_clk)) {
+ v4l2_err(&csi2->sd, "failed to get dphy clock\n");
+ ret = PTR_ERR(csi2->dphy_clk);
+ if (IS_ERR(csi2->pix_clk)) {
+ v4l2_err(&csi2->sd, "failed to get pixel clock\n");
+ ret = PTR_ERR(csi2->pix_clk);
+ if (!res) {
+ v4l2_err(&csi2->sd, "failed to get platform resources\n");
+ return -ENODEV;
+ }
+
+ csi2->base = devm_ioremap(&pdev->dev, res->start, PAGE_SIZE);
+ if (!csi2->base) {
+ v4l2_err(&csi2->sd, "failed to map CSI-2 registers\n");
+
+ platform_set_drvdata(pdev, &csi2->sd);
+
+ ret = v4l2_async_register_subdev(&csi2->sd);
+ if (ret)
+ goto rmmutex;
+ mutex_destroy(&csi2->lock);
+
+ v4l2_async_unregister_subdev(sd);
+ mutex_destroy(&csi2->lock);
+ { .compatible = "fsl,imx6-mipi-csi2", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, csi2_dt_ids);
+
+static struct platform_driver csi2_driver = {
+ .driver = {
+ .name = DEVICE_NAME,
+ .of_match_table = csi2_dt_ids,
+ },
+ .probe = csi2_probe,
+ .remove = csi2_remove,
+};
+
+module_platform_driver(csi2_driver);
+
+MODULE_DESCRIPTION("i.MX5/6 MIPI CSI-2 Receiver driver");
--
2.7.4
for sensors with a MIPI CSI2 interface.
drivers/staging/media/imx/Makefile | 1 +
drivers/staging/media/imx/imx6-mipi-csi2.c | 689 +++++++++++++++++++++++++++++
2 files changed, 690 insertions(+)
create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 878a126..3569625 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
new file mode 100644
index 0000000..1a71b40
--- /dev/null
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -0,0 +1,689 @@
+/*
+ * MIPI CSI-2 Receiver Subdev for Freescale i.MX6 SOC.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/clk.h>
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-of.h>
+#include <linux/platform_device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include "imx-media.h"
+
+/*
+ * there must be 5 pads: 1 input pad from sensor, and
+ * the 4 virtual channel output pads
+ */
+#define CSI2_SINK_PAD 0
+#define CSI2_NUM_SINK_PADS 1
+#define CSI2_NUM_SRC_PADS 4
+#define CSI2_NUM_PADS 5
+
+/*
+ * The default maximum bit-rate per lane in Mbps, if the
+ * source subdev does not provide V4L2_CID_LINK_FREQ.
+ */
+#define CSI2_DEFAULT_MAX_MBPS 849
+
+struct csi2_dev {
+ struct device *dev;
+ struct v4l2_subdev sd;
+ struct media_pad pad[CSI2_NUM_PADS];
+ struct v4l2_subdev sd;
+ struct clk *dphy_clk;
+ struct clk *pllref_clk;
+ struct clk *pix_clk; /* what is this? */
+ void __iomem *base;
+ struct v4l2_of_bus_mipi_csi2 bus;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ struct v4l2_mbus_framefmt format_mbus;
+ /* lock to protect all members below */
+ struct mutex lock;
+
+
+ int power_count;
+ bool stream_on;
+ struct v4l2_subdev *src_sd;
+ bool sink_linked[CSI2_NUM_SRC_PADS];
+};
+
+#define DEVICE_NAME "imx6-mipi-csi2"
+
+/* Register offsets */
+#define CSI2_VERSION 0x000
+#define CSI2_N_LANES 0x004
+#define CSI2_PHY_SHUTDOWNZ 0x008
+#define CSI2_DPHY_RSTZ 0x00c
+#define CSI2_RESETN 0x010
+#define CSI2_PHY_STATE 0x014
+#define PHY_STOPSTATEDATA_BIT 4
+#define PHY_STOPSTATEDATA(n) BIT(PHY_STOPSTATEDATA_BIT + (n))
+#define PHY_RXCLKACTIVEHS BIT(8)
+#define PHY_RXULPSCLKNOT BIT(9)
+#define PHY_STOPSTATECLK BIT(10)
+#define CSI2_DATA_IDS_1 0x018
+#define CSI2_DATA_IDS_2 0x01c
+#define CSI2_ERR1 0x020
+#define CSI2_ERR2 0x024
+#define CSI2_MSK1 0x028
+#define CSI2_MSK2 0x02c
+#define CSI2_PHY_TST_CTRL0 0x030
+#define PHY_TESTCLR BIT(0)
+#define PHY_TESTCLK BIT(1)
+#define CSI2_PHY_TST_CTRL1 0x034
+#define PHY_TESTEN BIT(16)
+#define CSI2_SFT_RESET 0xf00
+
+static inline struct csi2_dev *sd_to_dev(struct v4l2_subdev *sdev)
+{
+ return container_of(sdev, struct csi2_dev, sd);
+}
+
+static void csi2_enable(struct csi2_dev *csi2, bool enable)
+{
+ if (enable) {
+ writel(0x1, csi2->base + CSI2_PHY_SHUTDOWNZ);
+ writel(0x1, csi2->base + CSI2_DPHY_RSTZ);
+ writel(0x1, csi2->base + CSI2_RESETN);
+ } else {
+ writel(0x0, csi2->base + CSI2_PHY_SHUTDOWNZ);
+ writel(0x0, csi2->base + CSI2_DPHY_RSTZ);
+ writel(0x0, csi2->base + CSI2_RESETN);
+ }
+}
+
+static void csi2_set_lanes(struct csi2_dev *csi2)
+{
+ int lanes = csi2->bus.num_data_lanes;
+
+ writel(lanes - 1, csi2->base + CSI2_N_LANES);
+}
+
+static void dw_mipi_csi2_phy_write(struct csi2_dev *csi2,
+ u32 test_code, u32 test_data)
+{
+ /* Clear PHY test interface */
+ writel(PHY_TESTCLR, csi2->base + CSI2_PHY_TST_CTRL0);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Raise test interface strobe signal */
+ writel(PHY_TESTCLK, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Configure address write on falling edge and lower strobe signal */
+ writel(PHY_TESTEN | test_code, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Configure data write on rising edge and raise strobe signal */
+ writel(test_data, csi2->base + CSI2_PHY_TST_CTRL1);
+ writel(PHY_TESTCLK, csi2->base + CSI2_PHY_TST_CTRL0);
+
+ /* Clear strobe signal */
+ writel(0x0, csi2->base + CSI2_PHY_TST_CTRL0);
+}
+
+/*
+ * This table is based on the table documented at
+ * https://community.nxp.com/docs/DOC-94312. It assumes
+ * a 27MHz D-PHY pll reference clock.
+ */
+static const struct {
+ u32 max_mbps;
+ u32 hsfreqrange_sel;
+} hsfreq_map[] = {
+ { 90, 0x00}, {100, 0x20}, {110, 0x40}, {125, 0x02},
+ {140, 0x22}, {150, 0x42}, {160, 0x04}, {180, 0x24},
+ {200, 0x44}, {210, 0x06}, {240, 0x26}, {250, 0x46},
+ {270, 0x08}, {300, 0x28}, {330, 0x48}, {360, 0x2a},
+ {400, 0x4a}, {450, 0x0c}, {500, 0x2c}, {550, 0x0e},
+ {600, 0x2e}, {650, 0x10}, {700, 0x30}, {750, 0x12},
+ {800, 0x32}, {850, 0x14}, {900, 0x34}, {950, 0x54},
+ {1000, 0x74},
+};
+
+static int max_mbps_to_hsfreqrange_sel(u32 max_mbps)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(hsfreq_map); i++)
+ if (hsfreq_map[i].max_mbps > max_mbps)
+ return hsfreq_map[i].hsfreqrange_sel;
+
+ return -EINVAL;
+}
+
+static int csi2_dphy_init(struct csi2_dev *csi2)
+}
+
+{
+ struct v4l2_ctrl *ctrl;
+ u32 mbps_per_lane;
+ int sel;
+
+ ctrl = v4l2_ctrl_find(csi2->src_sd->ctrl_handler,
+ V4L2_CID_LINK_FREQ);
+ if (!ctrl)
+ mbps_per_lane = CSI2_DEFAULT_MAX_MBPS;
+ else
+ mbps_per_lane = DIV_ROUND_UP_ULL(2 * ctrl->qmenu_int[ctrl->val],
+ USEC_PER_SEC);
+
+ sel = max_mbps_to_hsfreqrange_sel(mbps_per_lane);
+ if (sel < 0)
+ return sel;
+
+ dw_mipi_csi2_phy_write(csi2, 0x44, sel);
+
+ return 0;
+}
+
+/*
+ return 0;
+}
+
+ * Waits for ultra-low-power state on D-PHY clock lane. This is currently
+ * unused and may not be needed at all, but keep around just in case.
+ */
+static int __maybe_unused csi2_dphy_wait_ulp(struct csi2_dev *csi2)
+{
+ u32 reg;
+ int ret;
+
+ /* wait for ULP on clock lane */
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ !(reg & PHY_RXULPSCLKNOT), 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "ULP timeout, phy_state = 0x%08x\n", reg);
+ return ret;
+ }
+
+ /* wait until no errors on bus */
+ }
+
+ ret = readl_poll_timeout(csi2->base + CSI2_ERR1, reg,
+ reg == 0x0, 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "stable bus timeout, err1 = 0x%08x\n", reg);
+ return ret;
+ }
+
+ return 0;
+ }
+
+}
+
+/* Waits for low-power LP-11 state on data and clock lanes. */
+static int csi2_dphy_wait_stopstate(struct csi2_dev *csi2)
+{
+ u32 mask, reg;
+ int ret;
+
+ mask = PHY_STOPSTATECLK |
+ ((csi2->bus.num_data_lanes - 1) << PHY_STOPSTATEDATA_BIT);
+
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ (reg & mask) == mask, 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "LP-11 timeout, phy_state = 0x%08x\n", reg);
+ return ret;
+ }
+
+ return 0;
+ }
+
+}
+
+/*
+ * Wait for active clock on the clock lane.
+ *
+ * FIXME: Currently unused, but it should be! It should be called
+ * from csi2_s_stream() below, at stream ON, but the required sequence
+ * of MIPI CSI-2 startup does not allow for an opportunity for this to
+ * be called. The sequence as specified in the i.MX6 reference manual
+ * is as follows:
+ *
+ * 1. Deassert presetn signal (global reset).
+ * It's not clear what this "global reset" signal is (maybe APB
+ * global reset), but in any case this step corresponds to
+ * csi2_s_power(ON) here.
+ *
+ * 2. Configure MIPI Camera Sensor to put all Tx lanes in PL-11 state.
+ * This must be carried out by the MIPI sensor's s_power(ON) subdev
+ * op.
+ *
+ * 3. D-PHY initialization.
+ * 4. CSI2 Controller programming (Set N_LANES, deassert PHY_SHUTDOWNZ,
+ * deassert PHY_RSTZ, deassert CSI2_RESETN).
+ * 5. Read the PHY status register (PHY_STATE) to confirm that all data and
+ * clock lanes of the D-PHY are in Stop State.
+ * These steps (3,4,5) are carried out by csi2_s_stream(ON) here.
+ *
+ * 6. Configure the MIPI Camera Sensor to start transmitting a clock on the
+ * D-PHY clock lane.
+ * This must be carried out by the MIPI sensor's s_stream(ON) subdev
+ * op.
+ *
+ * 7. CSI2 Controller programming - Read the PHY status register (PHY_STATE)
+ * to confirm that the D-PHY is receiving a clock on the D-PHY clock lane.
+ * This is implemented by this unused function, and _should_ be called
+ * by csi2_s_stream(ON) here, but csi2_s_stream(ON) has been taken up
+ * by steps 3,4,5 above already.
+ *
+ * In summary, a temporary solution would require a hard-coded delay in the
+ * MIPI sensor's s_stream(ON) op, to allow time for a stable clock lane.
+ *
+ * A longer term solution might be to create a new subdev op, perhaps
+ * called prepare_stream, that can be implemented here, and would be
+ * assigned steps 3,4,5. Then csi2_s_stream(ON) would become available
+ * as step 7.
+ */
+static int __maybe_unused csi2_dphy_wait_clock_lane(struct csi2_dev *csi2)
+{
+ u32 reg;
+ int ret;
+
+ ret = readl_poll_timeout(csi2->base + CSI2_PHY_STATE, reg,
+ (reg & PHY_RXCLKACTIVEHS), 0, 500000);
+ if (ret) {
+ v4l2_err(&csi2->sd, "clock lane timeout, phy_state = 0x%08x\n",
+ reg);
+ return ret;
+ }
+
+ return 0;
+ }
+
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+/* Startup Sequence Step 1 */
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int csi2_s_power(struct v4l2_subdev *sd, int on)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ int ret = 0;
+
+ mutex_lock(&csi2->lock);
+
+
+ /*
+ * If the power count is modified from 0 to != 0 or from != 0 to 0,
+ * update the power state.
+ */
+ if (csi2->power_count == !on) {
+ if (on) {
+ dev_dbg(csi2->dev, "power ON\n");
+ ret = clk_prepare_enable(csi2->pllref_clk);
+ if (ret)
+ goto out;
+ ret = clk_prepare_enable(csi2->dphy_clk);
+ goto out;
+ if (ret) {
+ clk_disable_unprepare(csi2->pllref_clk);
+ goto out;
+ }
+ } else {
+ dev_dbg(csi2->dev, "power OFF\n");
+ }
+ } else {
+ clk_disable_unprepare(csi2->dphy_clk);
+ clk_disable_unprepare(csi2->pllref_clk);
+ }
+ }
+
+ /* Update the power count. */
+ csi2->power_count += on ? 1 : -1;
+ WARN_ON(csi2->power_count < 0);
+out:
+ mutex_unlock(&csi2->lock);
+ return ret;
+}
+
+/* Startup Sequence Steps 3, 4, 5 */
+}
+
+static int csi2_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ int i, ret = 0;
+
+ mutex_lock(&csi2->lock);
+
+ if (!csi2->src_sd) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ for (i = 0; i < CSI2_NUM_SRC_PADS; i++) {
+ goto out;
+ }
+
+ if (csi2->sink_linked[i])
+ break;
+ }
+ if (i >= CSI2_NUM_SRC_PADS) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ if (enable && !csi2->stream_on) {
+ goto out;
+ }
+
+ dev_dbg(csi2->dev, "stream ON\n");
+
+ ret = clk_prepare_enable(csi2->pix_clk);
+ if (ret)
+ goto out;
+
+ /* Step 3 */
+ goto out;
+
+ ret = csi2_dphy_init(csi2);
+ if (ret) {
+ clk_disable_unprepare(csi2->pix_clk);
+ goto out;
+ }
+
+ /* Step 4 */
+ }
+
+ csi2_set_lanes(csi2);
+ csi2_enable(csi2, true);
+
+ /* Step 5 */
+ ret = csi2_dphy_wait_stopstate(csi2);
+ if (ret) {
+ csi2_enable(csi2, false);
+ clk_disable_unprepare(csi2->pix_clk);
+ goto out;
+ }
+ } else if (!enable && csi2->stream_on) {
+ dev_dbg(csi2->dev, "stream OFF\n");
+ csi2_enable(csi2, false);
+ clk_disable_unprepare(csi2->pix_clk);
+ }
+
+ csi2->stream_on = enable;
+out:
+ mutex_unlock(&csi2->lock);
+ return ret;
+}
+
+static int csi2_link_setup(struct media_entity *entity,
+}
+
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(csi2->dev, "link setup %s -> %s", remote->entity->name,
+ int ret = 0;
+
+ local->entity->name);
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ mutex_lock(&csi2->lock);
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+
+ if (local->flags & MEDIA_PAD_FL_SOURCE) {
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (csi2->sink_linked[local->index]) {
+ ret = -EBUSY;
+ goto out;
+ }
+ csi2->sink_linked[local->index] = true;
+ goto out;
+ }
+ } else {
+ csi2->sink_linked[local->index] = false;
+ }
+ } else {
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (csi2->src_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ csi2->src_sd = remote_sd;
+ goto out;
+ }
+ } else {
+ csi2->src_sd = NULL;
+ }
+ }
+
+out:
+ mutex_unlock(&csi2->lock);
+ return ret;
+}
+
+static int csi2_get_fmt(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *fmt;
+
+ mutex_lock(&csi2->lock);
+
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
+ fmt = v4l2_subdev_get_try_format(&csi2->sd, cfg,
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
+ sdformat->pad);
+ else
+ fmt = &csi2->format_mbus;
+
+ sdformat->format = *fmt;
+
+ mutex_unlock(&csi2->lock);
+ sdformat->format = *fmt;
+
+
+ return 0;
+}
+
+static int csi2_set_fmt(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ int ret = 0;
+
+ if (sdformat->pad >= CSI2_NUM_PADS)
+
+ return -EINVAL;
+
+ mutex_lock(&csi2->lock);
+
+ if (csi2->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ goto out;
+ }
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (sdformat->pad != CSI2_SINK_PAD)
+ sdformat->format = csi2->format_mbus;
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
+ cfg->try_fmt = sdformat->format;
+ else
+ csi2->format_mbus = sdformat->format;
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
+ cfg->try_fmt = sdformat->format;
+ else
+out:
+ mutex_unlock(&csi2->lock);
+ return ret;
+}
+
+static int csi2_link_validate(struct v4l2_subdev *sd,
+}
+
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ return v4l2_subdev_link_validate_frame_interval(link);
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int csi2_registered(struct v4l2_subdev *sd)
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ int i, ret;
+
+ for (i = 0; i < CSI2_NUM_PADS; i++) {
+
+ csi2->pad[i].flags = (i == CSI2_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+ }
+
+ }
+
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&csi2->format_mbus,
+ 640, 480, 0, V4L2_FIELD_NONE, NULL);
+ if (ret)
+ return ret;
+
+ return media_entity_pads_init(&sd->entity, CSI2_NUM_PADS, csi2->pad);
+ return ret;
+
+}
+
+static struct media_entity_operations csi2_entity_ops = {
+ .link_setup = csi2_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_core_ops csi2_core_ops = {
+};
+
+ .s_power = csi2_s_power,
+};
+
+static struct v4l2_subdev_video_ops csi2_video_ops = {
+ .s_stream = csi2_s_stream,
+};
+
+static struct v4l2_subdev_pad_ops csi2_pad_ops = {
+ .get_fmt = csi2_get_fmt,
+ .set_fmt = csi2_set_fmt,
+ .link_validate = csi2_link_validate,
+};
+
+static struct v4l2_subdev_ops csi2_subdev_ops = {
+ .core = &csi2_core_ops,
+ .video = &csi2_video_ops,
+ .pad = &csi2_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops csi2_internal_ops = {
+ .registered = csi2_registered,
+};
+
+static int csi2_parse_endpoints(struct csi2_dev *csi2)
+{
+ struct device_node *node = csi2->dev->of_node;
+ struct device_node *epnode;
+ struct v4l2_of_endpoint ep;
+
+ epnode = of_graph_get_endpoint_by_regs(node, 0, -1);
+ if (!epnode) {
+ v4l2_err(&csi2->sd, "failed to get sink endpoint node\n");
+ return -EINVAL;
+ }
+
+ v4l2_of_parse_endpoint(epnode, &ep);
+ }
+
+ of_node_put(epnode);
+
+ if (ep.bus_type != V4L2_MBUS_CSI2) {
+ v4l2_err(&csi2->sd, "invalid bus type, must be MIPI CSI2\n");
+ return -EINVAL;
+ }
+
+ csi2->bus = ep.bus.mipi_csi2;
+ }
+
+
+ dev_dbg(csi2->dev, "data lanes: %d\n", csi2->bus.num_data_lanes);
+ dev_dbg(csi2->dev, "flags: 0x%08x\n", csi2->bus.flags);
+ return 0;
+}
+
+static int csi2_probe(struct platform_device *pdev)
+}
+
+{
+ struct csi2_dev *csi2;
+ struct resource *res;
+ int ret;
+
+ csi2 = devm_kzalloc(&pdev->dev, sizeof(*csi2), GFP_KERNEL);
+ if (!csi2)
+ return -ENOMEM;
+
+ csi2->dev = &pdev->dev;
+
+ v4l2_subdev_init(&csi2->sd, &csi2_subdev_ops);
+ v4l2_set_subdevdata(&csi2->sd, &pdev->dev);
+ csi2->sd.internal_ops = &csi2_internal_ops;
+ csi2->sd.entity.ops = &csi2_entity_ops;
+ csi2->sd.dev = &pdev->dev;
+ csi2->sd.owner = THIS_MODULE;
+ csi2->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ strcpy(csi2->sd.name, DEVICE_NAME);
+ csi2->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+ csi2->sd.grp_id = IMX_MEDIA_GRP_ID_CSI2;
+
+ ret = csi2_parse_endpoints(csi2);
+ if (ret)
+ return ret;
+
+ csi2->pllref_clk = devm_clk_get(&pdev->dev, "ref");
+ return ret;
+
+ if (IS_ERR(csi2->pllref_clk)) {
+ v4l2_err(&csi2->sd, "failed to get pll reference clock\n");
+ ret = PTR_ERR(csi2->pllref_clk);
+ return ret;
+ }
+
+ csi2->dphy_clk = devm_clk_get(&pdev->dev, "dphy");
+ }
+
+ if (IS_ERR(csi2->dphy_clk)) {
+ v4l2_err(&csi2->sd, "failed to get dphy clock\n");
+ ret = PTR_ERR(csi2->dphy_clk);
+ return ret;
+ }
+
+ csi2->pix_clk = devm_clk_get(&pdev->dev, "pix");
+ }
+
+ if (IS_ERR(csi2->pix_clk)) {
+ v4l2_err(&csi2->sd, "failed to get pixel clock\n");
+ ret = PTR_ERR(csi2->pix_clk);
+ return ret;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ }
+
+ if (!res) {
+ v4l2_err(&csi2->sd, "failed to get platform resources\n");
+ return -ENODEV;
+ }
+
+ csi2->base = devm_ioremap(&pdev->dev, res->start, PAGE_SIZE);
+ if (!csi2->base) {
+ v4l2_err(&csi2->sd, "failed to map CSI-2 registers\n");
+ return -ENOMEM;
+ }
+
+ mutex_init(&csi2->lock);
+ }
+
+
+ platform_set_drvdata(pdev, &csi2->sd);
+
+ ret = v4l2_async_register_subdev(&csi2->sd);
+ if (ret)
+ goto rmmutex;
+
+ return 0;
+
+rmmutex:
+ return 0;
+
+ mutex_destroy(&csi2->lock);
+ return ret;
+}
+
+static int csi2_remove(struct platform_device *pdev)
+}
+
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+
+ v4l2_async_unregister_subdev(sd);
+ mutex_destroy(&csi2->lock);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct of_device_id csi2_dt_ids[] = {
+
+ return 0;
+}
+
+ { .compatible = "fsl,imx6-mipi-csi2", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, csi2_dt_ids);
+
+static struct platform_driver csi2_driver = {
+ .driver = {
+ .name = DEVICE_NAME,
+ .of_match_table = csi2_dt_ids,
+ },
+ .probe = csi2_probe,
+ .remove = csi2_remove,
+};
+
+module_platform_driver(csi2_driver);
+
+MODULE_DESCRIPTION("i.MX5/6 MIPI CSI-2 Receiver driver");
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
This is a media entity subdevice driver for the i.MX Video De-Interlacing
or Combining Block. So far this entity does not implement the Combining
function but only motion compensated deinterlacing. Video frames are
received from the CSI and are routed to the IC PRPVF entity.
Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
---
drivers/staging/media/imx/Makefile | 1 +
drivers/staging/media/imx/imx-media-vdic.c | 952 +++++++++++++++++++++++++++++
2 files changed, 953 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index c054490..1f01520 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,5 +4,6 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
new file mode 100644
index 0000000..0e7c24c
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -0,0 +1,952 @@
+/*
+ * V4L2 Deinterlacer Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2017 Mentor Graphics Inc.
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+
+/*
+ * This subdev implements two different video pipelines:
+ *
+ * CSI -> VDIC
+ *
+ * In this pipeline, the CSI sends a single interlaced field F(n-1)
+ * directly to the VDIC (and optionally the following field F(n)
+ * can be sent to memory via IDMAC channel 13). This pipeline only works
+ * in VDIC's high motion mode, which only requires a single field for
+ * processing. The other motion modes (low and medium) require three
+ * fields, so this pipeline does not work in those modes. Also, it is
+ * not clear how this pipeline can deal with the various field orders
+ * (sequential BT/TB, interlaced BT/TB).
+ *
+ * MEM -> CH8,9,10 -> VDIC
+ *
+ * In this pipeline, previous field F(n-1), current field F(n), and next
+ * field F(n+1) are transferred to the VDIC via IDMAC channels 8,9,10.
+ * These memory buffers can come from a video output or mem2mem device.
+ * All motion modes are supported by this pipeline.
+ *
+ * The "direct" CSI->VDIC pipeline requires no DMA, but it can only be
+ * used in high motion mode.
+ */
+
+struct vdic_priv;
+
+struct vdic_pipeline_ops {
+ int (*setup)(struct vdic_priv *priv);
+ void (*start)(struct vdic_priv *priv);
+ void (*stop)(struct vdic_priv *priv);
+ void (*disable)(struct vdic_priv *priv);
+};
+
+/*
+#define MAX_H_VDIC 2048
+ struct device *dev;
+ struct ipu_soc *ipu;
+ struct imx_media_dev *md;
+ struct v4l2_subdev sd;
+ struct media_pad pad[VDIC_NUM_PADS];
+ int ipu_id;
+
+ int active_input_pad;
+
+ struct ipuv3_channel *vdi_in_ch_p; /* F(n-1) transfer channel */
+ struct ipuv3_channel *vdi_in_ch; /* F(n) transfer channel */
+ struct ipuv3_channel *vdi_in_ch_n; /* F(n+1) transfer channel */
+
+ /* pipeline operations */
+ struct vdic_pipeline_ops *ops;
+
+ /* current and previous input buffers indirect path */
+ struct imx_media_buffer *curr_in_buf;
+ struct imx_media_buffer *prev_in_buf;
+
+ /*
+ * translated field type, input line stride, and field size
+ * for indirect path
+ */
+ u32 fieldtype;
+ u32 in_stride;
+ u32 field_size;
+
+ /* the source (a video device or subdev) */
+ struct media_entity *src;
+ /* the sink that will receive the progressive out buffers */
+ struct v4l2_subdev *sink_sd;
+
+ struct v4l2_mbus_framefmt format_mbus[VDIC_NUM_PADS];
+ const struct imx_media_pixfmt *cc[VDIC_NUM_PADS];
+
+ /* motion select control */
+ struct v4l2_ctrl_handler ctrl_hdlr;
+ enum ipu_motion_sel motion;
+{
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch_p))
+ ipu_idmac_put(priv->vdi_in_ch_p);
+ priv->vdi_in_ch_p = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch))
+ ipu_idmac_put(priv->vdi_in_ch);
+ priv->vdi_in_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch_n))
+ ipu_idmac_put(priv->vdi_in_ch_n);
+ priv->vdi_in_ch_n = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi))
+ ipu_vdi_put(priv->vdi);
+ priv->vdi = NULL;
+}
+
+static int vdic_get_ipu_resources(struct vdic_priv *priv)
+{
+ int ret, err_chan;
+
+ priv->ipu = priv->md->ipu[priv->ipu_id];
+
+ priv->vdi = ipu_vdi_get(priv->ipu);
+ if (IS_ERR(priv->vdi)) {
+ v4l2_err(&priv->sd, "failed to get VDIC\n");
+ ret = PTR_ERR(priv->vdi);
+ priv->vdi_in_ch_p = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_PREV);
+ if (IS_ERR(priv->vdi_in_ch_p)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_PREV;
+ ret = PTR_ERR(priv->vdi_in_ch_p);
+ goto out_err_chan;
+ }
+
+ priv->vdi_in_ch = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_CUR);
+ if (IS_ERR(priv->vdi_in_ch)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_CUR;
+ ret = PTR_ERR(priv->vdi_in_ch);
+ goto out_err_chan;
+ }
+
+ priv->vdi_in_ch_n = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_NEXT);
+ if (IS_ERR(priv->vdi_in_ch_n)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_NEXT;
+ ret = PTR_ERR(priv->vdi_in_ch_n);
+ goto out_err_chan;
+ }
+ v4l2_err(&priv->sd, "could not get IDMAC channel %u\n", err_chan);
+out:
+ vdic_put_ipu_resources(priv);
+ * output/mem2mem device at the IDMAC input pad sends us a new
+ * buffer. It kicks off the IDMAC read channels to bring in the
+ * buffer fields from memory and begin the conversions.
+ */
+static void __maybe_unused prepare_vdi_in_buffers(struct vdic_priv *priv,
+ struct imx_media_buffer *curr)
+{
+ dma_addr_t prev_phys, curr_phys, next_phys;
+ struct imx_media_buffer *prev;
+ struct vb2_buffer *curr_vb, *prev_vb;
+ u32 fs = priv->field_size;
+ u32 is = priv->in_stride;
+
+ /* current input buffer is now previous */
+ priv->prev_in_buf = priv->curr_in_buf;
+ priv->curr_in_buf = curr;
+ prev = priv->prev_in_buf ? priv->prev_in_buf : curr;
+
+ prev_vb = &prev->vbuf.vb2_buf;
+ curr_vb = &curr->vbuf.vb2_buf;
+
+ switch (priv->fieldtype) {
+ case V4L2_FIELD_SEQ_TB:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ break;
+ case V4L2_FIELD_SEQ_BT:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + fs;
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
+ break;
+ case V4L2_FIELD_INTERLACED_BT:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + is;
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
+ break;
+ default:
+ /* assume V4L2_FIELD_INTERLACED_TB */
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ break;
+ }
+
+ ipu_cpmem_set_buffer(priv->vdi_in_ch_p, 0, prev_phys);
+ ipu_cpmem_set_buffer(priv->vdi_in_ch, 0, curr_phys);
+ ipu_cpmem_set_buffer(priv->vdi_in_ch_n, 0, next_phys);
+
+ ipu_idmac_select_buffer(priv->vdi_in_ch_p, 0);
+ ipu_idmac_select_buffer(priv->vdi_in_ch, 0);
+ ipu_idmac_select_buffer(priv->vdi_in_ch_n, 0);
+}
+
+static int setup_vdi_channel(struct vdic_priv *priv,
+ struct ipuv3_channel *channel,
+ dma_addr_t phys0, dma_addr_t phys1)
+
+ image.pix.height /= 2;
+ image.phys1 = phys1;
+{
+ /* set VDIC to receive from CSI for direct path */
+ ipu_fsu_link(priv->ipu, IPUV3_CHANNEL_CSI_DIRECT,
+ IPUV3_CHANNEL_CSI_VDI_PREV);
+{
+}
+
+static void vdic_stop_direct(struct vdic_priv *priv)
+{
+}
+
+static void vdic_disable_direct(struct vdic_priv *priv)
+{
+ ipu_fsu_unlink(priv->ipu, IPUV3_CHANNEL_CSI_DIRECT,
+ IPUV3_CHANNEL_CSI_VDI_PREV);
+}
+
+static int vdic_setup_indirect(struct vdic_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt;
+ const struct imx_media_pixfmt *incc;
+ int in_size, ret;
+
+ infmt = &priv->format_mbus[VDIC_SINK_PAD_IDMAC];
+ incc = priv->cc[VDIC_SINK_PAD_IDMAC];
+
+ in_size = (infmt->width * incc->bpp * infmt->height) >> 3;
+
+ /* 1/2 full image size */
+ priv->field_size = in_size / 2;
+ priv->in_stride = incc->planar ?
+ infmt->width : (infmt->width * incc->bpp) >> 3;
+
+ priv->prev_in_buf = NULL;
+ priv->curr_in_buf = NULL;
+
+ priv->fieldtype = infmt->field;
+
+ /* init the vdi-in channels */
+ ret = setup_vdi_channel(priv, priv->vdi_in_ch_p, 0, 0);
+}
+
+static void vdic_start_indirect(struct vdic_priv *priv)
+ ipu_idmac_enable_channel(priv->vdi_in_ch);
+ ipu_idmac_enable_channel(priv->vdi_in_ch_n);
+}
+
+static void vdic_stop_indirect(struct vdic_priv *priv)
+{
+ /* disable channels */
+ ipu_idmac_disable_channel(priv->vdi_in_ch_p);
+ ipu_idmac_disable_channel(priv->vdi_in_ch);
+ ipu_idmac_disable_channel(priv->vdi_in_ch_n);
+}
+
+static void vdic_disable_indirect(struct vdic_priv *priv)
+{
+}
+
+static struct vdic_pipeline_ops direct_ops = {
+ .setup = vdic_setup_direct,
+ .start = vdic_start_direct,
+ .stop = vdic_stop_direct,
+ .disable = vdic_disable_direct,
+};
+
+static struct vdic_pipeline_ops indirect_ops = {
+ .setup = vdic_setup_indirect,
+ .start = vdic_start_indirect,
+ .stop = vdic_stop_indirect,
+ .disable = vdic_disable_indirect,
+};
+
+static int vdic_start(struct vdic_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt;
+ int ret;
+
+ infmt = &priv->format_mbus[priv->active_input_pad];
+
+ priv->ops = priv->csi_direct ? &direct_ops : &indirect_ops;
+
+ ret = vdic_get_ipu_resources(priv);
+ * init the VDIC.
+ *
+ * note we don't give infmt->code to ipu_vdi_setup(). The VDIC
+ * only supports 4:2:2 or 4:2:0, and this subdev will only
+ * negotiate 4:2:2 at its sink pads.
+ */
+ ipu_vdi_setup(priv->vdi, MEDIA_BUS_FMT_UYVY8_2X8,
+ infmt->width, infmt->height);
+ ipu_vdi_set_field_order(priv->vdi, V4L2_STD_UNKNOWN, infmt->field);
+ ipu_vdi_set_motion(priv->vdi, priv->motion);
+
+ ret = priv->ops->setup(priv);
+ if (ret)
+ goto out_put_ipu;
+
+ ipu_vdi_enable(priv->vdi);
+
+ priv->ops->start(priv);
+ vdic_put_ipu_resources(priv);
+{
+ priv->ops->stop(priv);
+ ipu_vdi_disable(priv->vdi);
+ priv->ops->disable(priv);
+
+ vdic_put_ipu_resources(priv);
+{
+ struct vdic_priv *priv = container_of(ctrl->handler,
+ struct vdic_priv, ctrl_hdlr);
+ enum ipu_motion_sel motion;
+ motion = ctrl->val;
+ if (motion != priv->motion) {
+ /* can't change motion control mid-streaming */
+ if (priv->stream_on) {
+ }
+ break;
+ default:
+ v4l2_err(&priv->sd, "Invalid control\n");
+ ret = -EINVAL;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ .s_ctrl = vdic_s_ctrl,
+};
+
+static const char * const vdic_ctrl_motion_menu[] = {
+ "No Motion Compensation",
+ "Low Motion",
+ "Medium Motion",
+ "High Motion",
+};
+
+static int vdic_init_controls(struct vdic_priv *priv)
+{
+
+ v4l2_ctrl_handler_init(hdlr, 1);
+
+ v4l2_ctrl_new_std_menu_items(hdlr, &vdic_ctrl_ops,
+ V4L2_CID_DEINTERLACING_MODE,
+ HIGH_MOTION, 0, HIGH_MOTION,
+ vdic_ctrl_motion_menu);
+
+ priv->sd.ctrl_handler = hdlr;
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+
+ if (!priv->src || !priv->sink_sd) {
+
+ else if (!enable && priv->stream_on)
+ vdic_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ struct v4l2_subdev_pad_config *cfg,
+ return -EINVAL;
+
+ if (code->pad == VDIC_SINK_PAD_IDMAC)
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, false);
+ struct v4l2_mbus_framefmt *fmt;
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __vdic_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+
+ if (sdformat->pad >= VDIC_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
+
+ switch (sdformat->pad) {
+ case VDIC_SRC_PAD_DIRECT:
+ infmt = __vdic_get_fmt(priv, cfg, priv->active_input_pad,
+ sdformat->which);
+
+ cc = imx_media_find_ipu_format(0, sdformat->format.code, false);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, false);
+ cc = imx_media_find_ipu_format(0, code, false);
+ sdformat->format.height = infmt->height;
+ /* output is always progressive! */
+ sdformat->format.field = V4L2_FIELD_NONE;
+ break;
+ case VDIC_SINK_PAD_IDMAC:
+ case VDIC_SINK_PAD_DIRECT:
+ if (sdformat->pad == VDIC_SINK_PAD_DIRECT) {
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0,
+ false);
+ cc = imx_media_find_ipu_format(0, code, false);
+ }
+
+ /* input must be interlaced! Choose SEQ_TB if not */
+ if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
+ sdformat->format.field = V4L2_FIELD_SEQ_TB;
+
+ mutex_lock(&priv->lock);
+ } else {
+ priv->sink_sd = NULL;
+ }
+
+
+ priv->src = NULL;
+ }
+ if (!vdev) {
+ ret = -ENODEV;
+ } else {
+ if (!(remote_sd->grp_id & IMX_MEDIA_GRP_ID_CSI) ||
+ remote->index != CSI_SRC_PAD_DIRECT) {
+ ret = -EINVAL;
+ }
+
+ priv->src = remote->entity;
+ /* record which input pad is now active */
+ priv->active_input_pad = local->index;
+out:
+ mutex_unlock(&priv->lock);
+
+ if (priv->csi_direct && priv->motion != HIGH_MOTION) {
+ v4l2_err(&priv->sd,
+ "direct CSI pipeline requires high motion\n");
+ ret = -EINVAL;
+ }
+
+ mutex_unlock(&priv->lock);
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ int i, ret;
+ for (i = 0; i < VDIC_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == VDIC_SRC_PAD_DIRECT) ?
+ MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+ code = 0;
+ if (i != VDIC_SINK_PAD_IDMAC)
+
+ ret = vdic_init_controls(priv);
+ if (ret)
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+}
+
+static struct v4l2_subdev_pad_ops vdic_pad_ops = {
+ .enum_mbus_code = vdic_enum_mbus_code,
+ .get_fmt = vdic_get_fmt,
+ .set_fmt = vdic_set_fmt,
+ .link_validate = vdic_link_validate,
+};
+
+static struct v4l2_subdev_video_ops vdic_video_ops = {
+ .s_stream = vdic_s_stream,
+ .g_frame_interval = vdic_g_frame_interval,
+ .s_frame_interval = vdic_s_frame_interval,
+};
+
+static struct media_entity_operations vdic_entity_ops = {
+ .link_setup = vdic_link_setup,
+ .video = &vdic_video_ops,
+ .pad = &vdic_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops vdic_internal_ops = {
+ .registered = vdic_registered,
+ .unregistered = vdic_unregistered,
+};
+
+static int imx_vdic_probe(struct platform_device *pdev)
+{
+ struct imx_media_internal_sd_platformdata *pdata;
+ struct vdic_priv *priv;
+ int ret;
+
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = &vdic_internal_ops;
+ priv->sd.entity.ops = &vdic_entity_ops;
+ priv->sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ /* get our group id */
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ goto free;
+
+ return 0;
+free:
+ mutex_destroy(&priv->lock);
+ mutex_destroy(&priv->lock);
+ { .name = "imx-ipuv3-vdic" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_vdic_ids);
+
+static struct platform_driver imx_vdic_driver = {
+ .probe = imx_vdic_probe,
+ .remove = imx_vdic_remove,
+ .id_table = imx_vdic_ids,
+ .driver = {
+ .name = "imx-ipuv3-vdic",
+ },
+};
+module_platform_driver(imx_vdic_driver);
+
+MODULE_DESCRIPTION("i.MX VDIC subdev driver");
+MODULE_ALIAS("platform:imx-ipuv3-vdic");
--
2.7.4
or Combining Block. So far this entity does not implement the Combining
function but only motion compensated deinterlacing. Video frames are
received from the CSI and are routed to the IC PRPVF entity.
Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
---
drivers/staging/media/imx/Makefile | 1 +
2 files changed, 953 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index c054490..1f01520 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,5 +4,6 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
new file mode 100644
index 0000000..0e7c24c
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -0,0 +1,952 @@
+/*
+ * V4L2 Deinterlacer Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+#include "imx-media.h"
+#include <media/v4l2-subdev.h>
+#include <media/imx.h>
+
+/*
+ * This subdev implements two different video pipelines:
+ *
+ * CSI -> VDIC
+ *
+ * In this pipeline, the CSI sends a single interlaced field F(n-1)
+ * directly to the VDIC (and optionally the following field F(n)
+ * can be sent to memory via IDMAC channel 13). This pipeline only works
+ * in VDIC's high motion mode, which only requires a single field for
+ * processing. The other motion modes (low and medium) require three
+ * fields, so this pipeline does not work in those modes. Also, it is
+ * not clear how this pipeline can deal with the various field orders
+ * (sequential BT/TB, interlaced BT/TB).
+ *
+ * MEM -> CH8,9,10 -> VDIC
+ *
+ * In this pipeline, previous field F(n-1), current field F(n), and next
+ * field F(n+1) are transferred to the VDIC via IDMAC channels 8,9,10.
+ * These memory buffers can come from a video output or mem2mem device.
+ * All motion modes are supported by this pipeline.
+ *
+ * The "direct" CSI->VDIC pipeline requires no DMA, but it can only be
+ * used in high motion mode.
+ */
+
+struct vdic_priv;
+
+struct vdic_pipeline_ops {
+ int (*setup)(struct vdic_priv *priv);
+ void (*start)(struct vdic_priv *priv);
+ void (*stop)(struct vdic_priv *priv);
+ void (*disable)(struct vdic_priv *priv);
+};
+
+/*
+ * Min/Max supported width and heights.
+ */
+#define MIN_W 176
+#define MIN_H 144
+#define MAX_W_VDIC 968
+ */
+#define MIN_W 176
+#define MIN_H 144
+#define MAX_H_VDIC 2048
+#define W_ALIGN 4 /* multiple of 16 pixels */
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct vdic_priv {
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+ struct device *dev;
+ struct ipu_soc *ipu;
+ struct imx_media_dev *md;
+ struct v4l2_subdev sd;
+ struct media_pad pad[VDIC_NUM_PADS];
+ int ipu_id;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ /* IPU units we require */
+ struct ipu_vdi *vdi;
+
+ int active_input_pad;
+
+ struct ipuv3_channel *vdi_in_ch_p; /* F(n-1) transfer channel */
+ struct ipuv3_channel *vdi_in_ch; /* F(n) transfer channel */
+ struct ipuv3_channel *vdi_in_ch_n; /* F(n+1) transfer channel */
+
+ /* pipeline operations */
+ struct vdic_pipeline_ops *ops;
+
+ /* current and previous input buffers indirect path */
+ struct imx_media_buffer *curr_in_buf;
+ struct imx_media_buffer *prev_in_buf;
+
+ /*
+ * translated field type, input line stride, and field size
+ * for indirect path
+ */
+ u32 fieldtype;
+ u32 in_stride;
+ u32 field_size;
+
+ /* the source (a video device or subdev) */
+ struct media_entity *src;
+ /* the sink that will receive the progressive out buffers */
+ struct v4l2_subdev *sink_sd;
+
+ struct v4l2_mbus_framefmt format_mbus[VDIC_NUM_PADS];
+ const struct imx_media_pixfmt *cc[VDIC_NUM_PADS];
+ struct v4l2_fract frame_interval;
+
+ /* the video device at IDMAC input pad */
+
+ struct imx_media_video_dev *vdev;
+
+ bool csi_direct; /* using direct CSI->VDIC->IC pipeline */
+
+
+ /* motion select control */
+ struct v4l2_ctrl_handler ctrl_hdlr;
+ enum ipu_motion_sel motion;
+
+ bool stream_on; /* streaming is on */
+};
+
+static void vdic_put_ipu_resources(struct vdic_priv *priv)
+ bool stream_on; /* streaming is on */
+};
+
+{
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch_p))
+ ipu_idmac_put(priv->vdi_in_ch_p);
+ priv->vdi_in_ch_p = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch))
+ ipu_idmac_put(priv->vdi_in_ch);
+ priv->vdi_in_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi_in_ch_n))
+ ipu_idmac_put(priv->vdi_in_ch_n);
+ priv->vdi_in_ch_n = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->vdi))
+ ipu_vdi_put(priv->vdi);
+ priv->vdi = NULL;
+}
+
+static int vdic_get_ipu_resources(struct vdic_priv *priv)
+{
+ int ret, err_chan;
+
+ priv->ipu = priv->md->ipu[priv->ipu_id];
+
+ priv->vdi = ipu_vdi_get(priv->ipu);
+ if (IS_ERR(priv->vdi)) {
+ v4l2_err(&priv->sd, "failed to get VDIC\n");
+ ret = PTR_ERR(priv->vdi);
+ goto out;
+ }
+
+ if (!priv->csi_direct) {
+ }
+
+ priv->vdi_in_ch_p = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_PREV);
+ if (IS_ERR(priv->vdi_in_ch_p)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_PREV;
+ ret = PTR_ERR(priv->vdi_in_ch_p);
+ goto out_err_chan;
+ }
+
+ priv->vdi_in_ch = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_CUR);
+ if (IS_ERR(priv->vdi_in_ch)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_CUR;
+ ret = PTR_ERR(priv->vdi_in_ch);
+ goto out_err_chan;
+ }
+
+ priv->vdi_in_ch_n = ipu_idmac_get(priv->ipu,
+ IPUV3_CHANNEL_MEM_VDI_NEXT);
+ if (IS_ERR(priv->vdi_in_ch_n)) {
+ err_chan = IPUV3_CHANNEL_MEM_VDI_NEXT;
+ ret = PTR_ERR(priv->vdi_in_ch_n);
+ goto out_err_chan;
+ }
+ }
+
+ return 0;
+
+out_err_chan:
+
+ return 0;
+
+ v4l2_err(&priv->sd, "could not get IDMAC channel %u\n", err_chan);
+out:
+ vdic_put_ipu_resources(priv);
+ return ret;
+}
+
+/*
+ * This function is currently unused, but will be called when the
+}
+
+/*
+ * output/mem2mem device at the IDMAC input pad sends us a new
+ * buffer. It kicks off the IDMAC read channels to bring in the
+ * buffer fields from memory and begin the conversions.
+ */
+static void __maybe_unused prepare_vdi_in_buffers(struct vdic_priv *priv,
+ struct imx_media_buffer *curr)
+{
+ dma_addr_t prev_phys, curr_phys, next_phys;
+ struct imx_media_buffer *prev;
+ struct vb2_buffer *curr_vb, *prev_vb;
+ u32 fs = priv->field_size;
+ u32 is = priv->in_stride;
+
+ /* current input buffer is now previous */
+ priv->prev_in_buf = priv->curr_in_buf;
+ priv->curr_in_buf = curr;
+ prev = priv->prev_in_buf ? priv->prev_in_buf : curr;
+
+ prev_vb = &prev->vbuf.vb2_buf;
+ curr_vb = &curr->vbuf.vb2_buf;
+
+ switch (priv->fieldtype) {
+ case V4L2_FIELD_SEQ_TB:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ break;
+ case V4L2_FIELD_SEQ_BT:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + fs;
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
+ break;
+ case V4L2_FIELD_INTERLACED_BT:
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + is;
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
+ break;
+ default:
+ /* assume V4L2_FIELD_INTERLACED_TB */
+ prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
+ curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
+ next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
+ break;
+ }
+
+ ipu_cpmem_set_buffer(priv->vdi_in_ch_p, 0, prev_phys);
+ ipu_cpmem_set_buffer(priv->vdi_in_ch, 0, curr_phys);
+ ipu_cpmem_set_buffer(priv->vdi_in_ch_n, 0, next_phys);
+
+ ipu_idmac_select_buffer(priv->vdi_in_ch_p, 0);
+ ipu_idmac_select_buffer(priv->vdi_in_ch, 0);
+ ipu_idmac_select_buffer(priv->vdi_in_ch_n, 0);
+}
+
+static int setup_vdi_channel(struct vdic_priv *priv,
+ struct ipuv3_channel *channel,
+ dma_addr_t phys0, dma_addr_t phys1)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_video_dev *vdev = priv->vdev;
+ unsigned int burst_size;
+ struct ipu_image image;
+ int ret;
+ struct ipu_image image;
+
+ ipu_cpmem_zero(channel);
+
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ /* one field to VDIC channels */
+
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ image.pix.height /= 2;
+ image.rect.width = image.pix.width;
+ image.rect.height = image.pix.height;
+ image.phys0 = phys0;
+ image.rect.height = image.pix.height;
+ image.phys1 = phys1;
+
+ ret = ipu_cpmem_set_image(channel, &image);
+ ret = ipu_cpmem_set_image(channel, &image);
+ if (ret)
+ return ret;
+
+ return ret;
+
+ burst_size = (image.pix.width & 0xf) ? 8 : 16;
+ ipu_cpmem_set_burstsize(channel, burst_size);
+
+ ipu_cpmem_set_axi_id(channel, 1);
+
+ ipu_idmac_set_double_buffer(channel, false);
+ ipu_cpmem_set_axi_id(channel, 1);
+
+
+ return 0;
+}
+
+static int vdic_setup_direct(struct vdic_priv *priv)
+ return 0;
+}
+
+{
+ /* set VDIC to receive from CSI for direct path */
+ ipu_fsu_link(priv->ipu, IPUV3_CHANNEL_CSI_DIRECT,
+ IPUV3_CHANNEL_CSI_VDI_PREV);
+
+ return 0;
+}
+
+static void vdic_start_direct(struct vdic_priv *priv)
+ return 0;
+}
+
+{
+}
+
+static void vdic_stop_direct(struct vdic_priv *priv)
+{
+}
+
+static void vdic_disable_direct(struct vdic_priv *priv)
+{
+ ipu_fsu_unlink(priv->ipu, IPUV3_CHANNEL_CSI_DIRECT,
+ IPUV3_CHANNEL_CSI_VDI_PREV);
+}
+
+static int vdic_setup_indirect(struct vdic_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt;
+ const struct imx_media_pixfmt *incc;
+ int in_size, ret;
+
+ infmt = &priv->format_mbus[VDIC_SINK_PAD_IDMAC];
+ incc = priv->cc[VDIC_SINK_PAD_IDMAC];
+
+ in_size = (infmt->width * incc->bpp * infmt->height) >> 3;
+
+ /* 1/2 full image size */
+ priv->field_size = in_size / 2;
+ priv->in_stride = incc->planar ?
+ infmt->width : (infmt->width * incc->bpp) >> 3;
+
+ priv->prev_in_buf = NULL;
+ priv->curr_in_buf = NULL;
+
+ priv->fieldtype = infmt->field;
+
+ /* init the vdi-in channels */
+ ret = setup_vdi_channel(priv, priv->vdi_in_ch_p, 0, 0);
+ if (ret)
+ return ret;
+ ret = setup_vdi_channel(priv, priv->vdi_in_ch, 0, 0);
+ return ret;
+ if (ret)
+ return ret;
+ return setup_vdi_channel(priv, priv->vdi_in_ch_n, 0, 0);
+ return ret;
+}
+
+static void vdic_start_indirect(struct vdic_priv *priv)
+{
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->vdi_in_ch_p);
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->vdi_in_ch);
+ ipu_idmac_enable_channel(priv->vdi_in_ch_n);
+}
+
+static void vdic_stop_indirect(struct vdic_priv *priv)
+{
+ /* disable channels */
+ ipu_idmac_disable_channel(priv->vdi_in_ch_p);
+ ipu_idmac_disable_channel(priv->vdi_in_ch);
+ ipu_idmac_disable_channel(priv->vdi_in_ch_n);
+}
+
+static void vdic_disable_indirect(struct vdic_priv *priv)
+{
+}
+
+static struct vdic_pipeline_ops direct_ops = {
+ .setup = vdic_setup_direct,
+ .start = vdic_start_direct,
+ .stop = vdic_stop_direct,
+ .disable = vdic_disable_direct,
+};
+
+static struct vdic_pipeline_ops indirect_ops = {
+ .setup = vdic_setup_indirect,
+ .start = vdic_start_indirect,
+ .stop = vdic_stop_indirect,
+ .disable = vdic_disable_indirect,
+};
+
+static int vdic_start(struct vdic_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt;
+ int ret;
+
+ infmt = &priv->format_mbus[priv->active_input_pad];
+
+ priv->ops = priv->csi_direct ? &direct_ops : &indirect_ops;
+
+ ret = vdic_get_ipu_resources(priv);
+ if (ret)
+ return ret;
+
+ /*
+ return ret;
+
+ * init the VDIC.
+ *
+ * note we don't give infmt->code to ipu_vdi_setup(). The VDIC
+ * only supports 4:2:2 or 4:2:0, and this subdev will only
+ * negotiate 4:2:2 at its sink pads.
+ */
+ ipu_vdi_setup(priv->vdi, MEDIA_BUS_FMT_UYVY8_2X8,
+ infmt->width, infmt->height);
+ ipu_vdi_set_field_order(priv->vdi, V4L2_STD_UNKNOWN, infmt->field);
+ ipu_vdi_set_motion(priv->vdi, priv->motion);
+
+ ret = priv->ops->setup(priv);
+ if (ret)
+ goto out_put_ipu;
+
+ ipu_vdi_enable(priv->vdi);
+
+ priv->ops->start(priv);
+
+ return 0;
+
+out_put_ipu:
+ return 0;
+
+ vdic_put_ipu_resources(priv);
+ return ret;
+}
+
+static void vdic_stop(struct vdic_priv *priv)
+}
+
+{
+ priv->ops->stop(priv);
+ ipu_vdi_disable(priv->vdi);
+ priv->ops->disable(priv);
+
+ vdic_put_ipu_resources(priv);
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int vdic_s_ctrl(struct v4l2_ctrl *ctrl)
+
+/*
+ * V4L2 subdev operations.
+ */
+
+{
+ struct vdic_priv *priv = container_of(ctrl->handler,
+ struct vdic_priv, ctrl_hdlr);
+ enum ipu_motion_sel motion;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+
+ switch (ctrl->id) {
+ case V4L2_CID_DEINTERLACING_MODE:
+ switch (ctrl->id) {
+ motion = ctrl->val;
+ if (motion != priv->motion) {
+ /* can't change motion control mid-streaming */
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->motion = motion;
+ goto out;
+ }
+ }
+ break;
+ default:
+ v4l2_err(&priv->sd, "Invalid control\n");
+ ret = -EINVAL;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops vdic_ctrl_ops = {
+}
+
+ .s_ctrl = vdic_s_ctrl,
+};
+
+static const char * const vdic_ctrl_motion_menu[] = {
+ "No Motion Compensation",
+ "Low Motion",
+ "Medium Motion",
+ "High Motion",
+};
+
+static int vdic_init_controls(struct vdic_priv *priv)
+{
+ struct v4l2_ctrl_handler *hdlr = &priv->ctrl_hdlr;
+ int ret;
+
+ v4l2_ctrl_handler_init(hdlr, 1);
+
+ v4l2_ctrl_new_std_menu_items(hdlr, &vdic_ctrl_ops,
+ V4L2_CID_DEINTERLACING_MODE,
+ HIGH_MOTION, 0, HIGH_MOTION,
+ vdic_ctrl_motion_menu);
+
+ priv->sd.ctrl_handler = hdlr;
+
+ if (hdlr->error) {
+ ret = hdlr->error;
+ goto out_free;
+ }
+
+ v4l2_ctrl_handler_setup(hdlr);
+ return 0;
+
+out_free:
+ v4l2_ctrl_handler_free(hdlr);
+ if (hdlr->error) {
+ ret = hdlr->error;
+ goto out_free;
+ }
+
+ v4l2_ctrl_handler_setup(hdlr);
+ return 0;
+
+out_free:
+ v4l2_ctrl_handler_free(hdlr);
+ return ret;
+}
+
+static int vdic_s_stream(struct v4l2_subdev *sd, int enable)
+}
+
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+
+ if (!priv->src || !priv->sink_sd) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ dev_dbg(priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+ goto out;
+ }
+
+
+ if (enable && !priv->stream_on)
+ ret = vdic_start(priv);
+ else if (!enable && priv->stream_on)
+ vdic_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= VDIC_NUM_PADS)
+{
+ return -EINVAL;
+
+ if (code->pad == VDIC_SINK_PAD_IDMAC)
+ return imx_media_enum_format(NULL, &code->code, code->index,
+ false, false);
+
+ return imx_media_enum_ipu_format(NULL, &code->code, code->index, false);
+}
+
+static struct v4l2_mbus_framefmt *
+__vdic_get_fmt(struct vdic_priv *priv, struct v4l2_subdev_pad_config *cfg,
+
+static struct v4l2_mbus_framefmt *
+ unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+{
+ if (which == V4L2_SUBDEV_FORMAT_TRY)
+ return v4l2_subdev_get_try_format(&priv->sd, cfg, pad);
+ else
+ return &priv->format_mbus[pad];
+}
+
+static int vdic_get_fmt(struct v4l2_subdev *sd,
+ return &priv->format_mbus[pad];
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= VDIC_NUM_PADS)
+
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ fmt = __vdic_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
+ if (!fmt) {
+ ret = -EINVAL;
+ ret = -EINVAL;
+ goto out;
+ }
+
+ }
+
+ sdformat->format = *fmt;
+out:
+ mutex_unlock(&priv->lock);
+out:
+ return ret;
+}
+
+static int vdic_set_fmt(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ const struct imx_media_pixfmt *cc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= VDIC_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W_VDIC,
+ goto out;
+ }
+
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
+
+ switch (sdformat->pad) {
+ case VDIC_SRC_PAD_DIRECT:
+ infmt = __vdic_get_fmt(priv, cfg, priv->active_input_pad,
+ sdformat->which);
+
+ cc = imx_media_find_ipu_format(0, sdformat->format.code, false);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, false);
+ cc = imx_media_find_ipu_format(0, code, false);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ sdformat->format.width = infmt->width;
+ }
+
+ sdformat->format.height = infmt->height;
+ /* output is always progressive! */
+ sdformat->format.field = V4L2_FIELD_NONE;
+ break;
+ case VDIC_SINK_PAD_IDMAC:
+ case VDIC_SINK_PAD_DIRECT:
+ if (sdformat->pad == VDIC_SINK_PAD_DIRECT) {
+ cc = imx_media_find_ipu_format(0, sdformat->format.code,
+ false);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0,
+ false);
+ cc = imx_media_find_ipu_format(0, code, false);
+ sdformat->format.code = cc->codes[0];
+ }
+ } else {
+ }
+ cc = imx_media_find_format(0, sdformat->format.code,
+ false, false);
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0,
+ false, false);
+ imx_media_enum_format(NULL, &code, 0,
+ cc = imx_media_find_format(0, code,
+ false, false);
+ sdformat->format.code = cc->codes[0];
+ }
+ }
+
+ /* input must be interlaced! Choose SEQ_TB if not */
+ if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
+ sdformat->format.field = V4L2_FIELD_SEQ_TB;
+ break;
+ default:
+ ret = -EINVAL;
+ default:
+ ret = -EINVAL;
+ goto out;
+ }
+
+ }
+
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ priv->cc[sdformat->pad] = cc;
+ }
+
+out:
+ return ret;
+}
+
+static int vdic_g_frame_interval(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+{
+
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+ mutex_lock(&priv->lock);
+ fi->interval = priv->frame_interval;
+
+ return 0;
+}
+
+static int vdic_s_frame_interval(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+{
+
+ mutex_lock(&priv->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad == VDIC_SRC_PAD_DIRECT)
+ /* Output pads mirror active input pad, no limits on input pads */
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+ mutex_unlock(&priv->lock);
+
+ priv->frame_interval = fi->interval;
+
+
+ return 0;
+}
+
+static int vdic_link_setup(struct media_entity *entity,
+ return 0;
+}
+
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(priv->dev, "link setup %s -> %s", remote->entity->name,
+ int ret = 0;
+
+ local->entity->name);
+
+ mutex_lock(&priv->lock);
+
+
+ if (local->flags & MEDIA_PAD_FL_SOURCE) {
+ if (local->flags & MEDIA_PAD_FL_SOURCE) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ ret = -EINVAL;
+ goto out;
+ }
+
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->sink_sd) {
+ ret = -EBUSY;
+ goto out;
+ }
+ priv->sink_sd = remote_sd;
+ goto out;
+ }
+ } else {
+ priv->sink_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is a sink pad */
+ }
+
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src) {
+ ret = -EBUSY;
+ goto out;
+ }
+ } else {
+ goto out;
+ }
+ priv->src = NULL;
+ goto out;
+ }
+
+ if (local->index == VDIC_SINK_PAD_IDMAC) {
+ }
+
+ struct imx_media_video_dev *vdev = priv->vdev;
+
+
+ if (!is_media_entity_v4l2_video_device(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ ret = -EINVAL;
+ }
+ if (!vdev) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ priv->csi_direct = false;
+ }
+
+ } else {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ ret = -EINVAL;
+ goto out;
+ }
+
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ /* direct pad must connect to a CSI */
+
+ if (!(remote_sd->grp_id & IMX_MEDIA_GRP_ID_CSI) ||
+ remote->index != CSI_SRC_PAD_DIRECT) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ priv->csi_direct = true;
+ }
+
+ }
+
+ priv->src = remote->entity;
+ /* record which input pad is now active */
+ priv->active_input_pad = local->index;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int vdic_link_validate(struct v4l2_subdev *sd,
+}
+
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ if (ret)
+ return ret;
+
+ mutex_lock(&priv->lock);
+ return ret;
+
+
+ if (priv->csi_direct && priv->motion != HIGH_MOTION) {
+ v4l2_err(&priv->sd,
+ "direct CSI pipeline requires high motion\n");
+ ret = -EINVAL;
+ }
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+/*
+}
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int vdic_registered(struct v4l2_subdev *sd)
+ */
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ int i, ret;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+ for (i = 0; i < VDIC_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == VDIC_SRC_PAD_DIRECT) ?
+ MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+ code = 0;
+ if (i != VDIC_SINK_PAD_IDMAC)
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+
+ /* set a default mbus format */
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+ }
+
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+ priv->active_input_pad = VDIC_SINK_PAD_DIRECT;
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+
+
+ ret = vdic_init_controls(priv);
+ if (ret)
+ return ret;
+
+ ret = media_entity_pads_init(&sd->entity, VDIC_NUM_PADS, priv->pad);
+ return ret;
+
+ if (ret)
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+
+ return ret;
+}
+
+static void vdic_unregistered(struct v4l2_subdev *sd)
+}
+
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+}
+
+static struct v4l2_subdev_pad_ops vdic_pad_ops = {
+ .enum_mbus_code = vdic_enum_mbus_code,
+ .get_fmt = vdic_get_fmt,
+ .set_fmt = vdic_set_fmt,
+ .link_validate = vdic_link_validate,
+};
+
+static struct v4l2_subdev_video_ops vdic_video_ops = {
+ .s_stream = vdic_s_stream,
+ .g_frame_interval = vdic_g_frame_interval,
+ .s_frame_interval = vdic_s_frame_interval,
+};
+
+static struct media_entity_operations vdic_entity_ops = {
+ .link_setup = vdic_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_ops vdic_subdev_ops = {
+};
+
+ .video = &vdic_video_ops,
+ .pad = &vdic_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops vdic_internal_ops = {
+ .registered = vdic_registered,
+ .unregistered = vdic_unregistered,
+};
+
+static int imx_vdic_probe(struct platform_device *pdev)
+{
+ struct imx_media_internal_sd_platformdata *pdata;
+ struct vdic_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ if (!priv)
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ pdata = priv->dev->platform_data;
+ priv->ipu_id = pdata->ipu_id;
+
+ v4l2_subdev_init(&priv->sd, &vdic_subdev_ops);
+ priv->ipu_id = pdata->ipu_id;
+
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = &vdic_internal_ops;
+ priv->sd.entity.ops = &vdic_entity_ops;
+ priv->sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ /* get our group id */
+ priv->sd.grp_id = pdata->grp_id;
+ strncpy(priv->sd.name, pdata->sd_name, sizeof(priv->sd.name));
+
+ mutex_init(&priv->lock);
+ strncpy(priv->sd.name, pdata->sd_name, sizeof(priv->sd.name));
+
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ goto free;
+
+ return 0;
+free:
+ mutex_destroy(&priv->lock);
+ return ret;
+}
+
+static int imx_vdic_remove(struct platform_device *pdev)
+}
+
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+
+ v4l2_info(sd, "Removing\n");
+
+ v4l2_async_unregister_subdev(sd);
+ v4l2_info(sd, "Removing\n");
+
+ mutex_destroy(&priv->lock);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct platform_device_id imx_vdic_ids[] = {
+
+ return 0;
+}
+
+ { .name = "imx-ipuv3-vdic" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_vdic_ids);
+
+static struct platform_driver imx_vdic_driver = {
+ .probe = imx_vdic_probe,
+ .remove = imx_vdic_remove,
+ .id_table = imx_vdic_ids,
+ .driver = {
+ .name = "imx-ipuv3-vdic",
+ },
+};
+module_platform_driver(imx_vdic_driver);
+
+MODULE_DESCRIPTION("i.MX VDIC subdev driver");
+MODULE_ALIAS("platform:imx-ipuv3-vdic");
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
This adds a header file for use by userspace programs wanting to interact
with the i.MX media driver. It defines custom v4l2 controls for the
i.MX v4l2 subdevices.
include/uapi/media/Kbuild | 1 +
include/uapi/media/imx.h | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+)
create mode 100644 include/uapi/media/imx.h
diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
index aafaa5a..fa78958 100644
--- a/include/uapi/media/Kbuild
+++ b/include/uapi/media/Kbuild
@@ -1 +1,2 @@
# UAPI Header export list
+header-y += imx.h
diff --git a/include/uapi/media/imx.h b/include/uapi/media/imx.h
new file mode 100644
index 0000000..f573de4
--- /dev/null
+++ b/include/uapi/media/imx.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (c) 2014-2015 Mentor Graphics Inc.
+ * Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version
+ */
+
+#ifndef __UAPI_MEDIA_IMX_H__
+#define __UAPI_MEDIA_IMX_H__
+
+enum imx_ctrl_id {
+ V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE + 0),
+ V4L2_CID_IMX_FIM_NUM,
+ V4L2_CID_IMX_FIM_TOLERANCE_MIN,
+ V4L2_CID_IMX_FIM_TOLERANCE_MAX,
+ V4L2_CID_IMX_FIM_NUM_SKIP,
+};
+
+#endif
--
2.7.4
with the i.MX media driver. It defines custom v4l2 controls for the
i.MX v4l2 subdevices.
include/uapi/media/Kbuild | 1 +
include/uapi/media/imx.h | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+)
create mode 100644 include/uapi/media/imx.h
diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
index aafaa5a..fa78958 100644
--- a/include/uapi/media/Kbuild
+++ b/include/uapi/media/Kbuild
@@ -1 +1,2 @@
# UAPI Header export list
+header-y += imx.h
diff --git a/include/uapi/media/imx.h b/include/uapi/media/imx.h
new file mode 100644
index 0000000..f573de4
--- /dev/null
+++ b/include/uapi/media/imx.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (c) 2014-2015 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by the
+ * This program is free software; you can redistribute it and/or modify
+ * Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version
+ */
+
+#ifndef __UAPI_MEDIA_IMX_H__
+#define __UAPI_MEDIA_IMX_H__
+
+enum imx_ctrl_id {
+ V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE + 0),
+ V4L2_CID_IMX_FIM_NUM,
+ V4L2_CID_IMX_FIM_TOLERANCE_MIN,
+ V4L2_CID_IMX_FIM_TOLERANCE_MAX,
+ V4L2_CID_IMX_FIM_NUM_SKIP,
+};
+
+#endif
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
As part of this, separate format try code from *_set_fmt() into
*_try_fmt(), so that the latter function can be used to propagate
a legal format from sink to source. This also reduces subsequent
bloat in *_set_fmt().
imx-ic-prp never needed separate formats for sink and source pads,
so propagation in this case was easy, just have only a single
format shared by both pads.
drivers/staging/media/imx/imx-ic-prp.c | 31 +++---
drivers/staging/media/imx/imx-ic-prpencvf.c | 86 ++++++++++-----
drivers/staging/media/imx/imx-media-capture.c | 12 ++
drivers/staging/media/imx/imx-media-csi.c | 152 ++++++++++++++++----------
drivers/staging/media/imx/imx-media-vdic.c | 72 +++++++-----
5 files changed, 224 insertions(+), 129 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 83cd2b4..ec742e6 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -56,8 +56,7 @@ struct prp_priv {
- struct v4l2_mbus_framefmt format_mbus[PRP_NUM_PADS];
- const struct imx_media_pixfmt *cc[PRP_NUM_PADS];
+ struct v4l2_mbus_framefmt format_mbus;
struct v4l2_fract frame_interval;
bool stream_on; /* streaming is on */
@@ -98,7 +97,7 @@ __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
if (which == V4L2_SUBDEV_FORMAT_TRY)
- return &priv->format_mbus[pad];
+ return &priv->format_mbus;
}
/*
@@ -167,7 +166,7 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sdformat)
{
+ const struct imx_media_pixfmt *cc;
struct v4l2_mbus_framefmt *infmt;
int ret = 0;
u32 code;
@@ -201,17 +200,14 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
- cc = imx_media_find_ipu_format(infmt->code, CS_SEL_ANY);
sdformat->format = *infmt;
break;
}
- if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
- }
+ else
+ priv->format_mbus = sdformat->format;
out:
mutex_unlock(&priv->lock);
@@ -427,20 +423,19 @@ static int prp_registered(struct v4l2_subdev *sd)
for (i = 0; i < PRP_NUM_PADS; i++) {
priv->pad[i].flags = (i == PRP_SINK_PAD) ?
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
-
- /* set a default mbus format */
- imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
- ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
- 640, 480, code, V4L2_FIELD_NONE,
- &priv->cc[i]);
- if (ret)
- return ret;
priv->frame_interval.denominator = 30;
+ /* set a default mbus format */
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus, 640, 480, code,
+ V4L2_FIELD_NONE, NULL);
index b42103c..644dd33 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -753,35 +753,23 @@ static int prp_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int prp_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void prp_try_fmt(struct prp_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ const struct imx_media_pixfmt **cc)
{
- struct prp_priv *priv = sd_to_priv(sd);
- const struct imx_media_pixfmt *cc;
- struct v4l2_mbus_framefmt *infmt;
- int ret = 0;
- u32 code;
-
- if (sdformat->pad >= PRPENCVF_NUM_PADS)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
+ *cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
+ if (!*cc) {
+ u32 code;
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
-
- cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
- if (!cc) {
imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
- cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ sdformat->format.code = (*cc)->codes[0];
+ struct v4l2_mbus_framefmt *infmt =
+ __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
sdformat->which);
if (sdformat->format.field != V4L2_FIELD_NONE)
@@ -809,14 +797,60 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
}
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
+ if (sdformat->pad == PRPENCVF_SINK_PAD) {
+
+ format.pad = PRPENCVF_SRC_PAD;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ prp_try_fmt(priv, cfg, &format, &outcc);
+
+ priv->format_mbus[PRPENCVF_SRC_PAD] = format.format;
+ priv->cc[PRPENCVF_SRC_PAD] = outcc;
}
+ /* propagate output pad format to capture device */
+ imx_media_mbus_fmt_to_pix_fmt(&vdev_fmt,
+ &priv->format_mbus[PRPENCVF_SRC_PAD],
+ priv->cc[PRPENCVF_SRC_PAD]);
+ mutex_unlock(&priv->lock);
+ imx_media_capture_device_set_format(vdev, &vdev_fmt);
+
+ return 0;
out:
mutex_unlock(&priv->lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index a757e05..ee91439 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -498,6 +498,18 @@ static struct video_device capture_videodev = {
.device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
};
+void imx_media_capture_device_set_format(struct imx_media_video_dev *vdev,
+ struct v4l2_pix_format *pix)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+
+ mutex_lock(&priv->mutex);
+ priv->vdev.fmt.fmt.pix = *pix;
+ priv->vdev.cc = imx_media_find_format(pix->pixelformat, CS_SEL_ANY);
+ mutex_unlock(&priv->mutex);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_set_format);
+
struct imx_media_buffer *
imx_media_capture_device_next_buf(struct imx_media_video_dev *vdev)
{
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index cf070be..fc0036a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -986,11 +986,11 @@ __csi_get_fmt(struct csi_priv *priv, struct v4l2_subdev_pad_config *cfg,
return &priv->format_mbus[pad];
}
-static int csi_try_crop(struct csi_priv *priv,
- struct v4l2_rect *crop,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_mbus_framefmt *infmt,
- struct imx_media_subdev *sensor)
+static void csi_try_crop(struct csi_priv *priv,
+ struct v4l2_rect *crop,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_mbus_framefmt *infmt,
+ struct imx_media_subdev *sensor)
{
struct v4l2_of_endpoint *sensor_ep;
@@ -1019,8 +1019,6 @@ static int csi_try_crop(struct csi_priv *priv,
if (crop->top + crop->height > infmt->height)
crop->top = infmt->height - crop->height;
}
-
- return 0;
}
static int csi_enum_mbus_code(struct v4l2_subdev *sd,
@@ -1092,34 +1090,17 @@ static int csi_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int csi_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void csi_try_fmt(struct csi_priv *priv,
+ struct imx_media_subdev *sensor,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ struct v4l2_rect *crop,
+ const struct imx_media_pixfmt **cc)
{
- struct csi_priv *priv = v4l2_get_subdevdata(sd);
- const struct imx_media_pixfmt *cc, *incc;
- struct imx_media_subdev *sensor;
- struct v4l2_rect crop;
- int ret = 0;
u32 code;
- if (sdformat->pad >= CSI_NUM_PADS)
- return -EINVAL;
-
- sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
- if (IS_ERR(sensor)) {
- v4l2_err(&priv->sd, "no sensor attached\n");
- return PTR_ERR(sensor);
- }
-
- mutex_lock(&priv->lock);
-
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
-
if (incc->bayer) {
sdformat->format.code = infmt->code;
- cc = incc;
+ *cc = incc;
} else {
u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
CS_SEL_YUV : CS_SEL_RGB;
- cc = imx_media_find_ipu_format(sdformat->format.code,
- cs_sel);
- if (!cc) {
+ *cc = imx_media_find_ipu_format(sdformat->format.code,
+ cs_sel);
+ if (!*cc) {
imx_media_enum_ipu_format(&code, 0, cs_sel);
- cc = imx_media_find_ipu_format(code, cs_sel);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, cs_sel);
+ sdformat->format.code = (*cc)->codes[0];
}
}
@@ -1172,39 +1153,92 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
W_ALIGN, &sdformat->format.height,
MIN_H, MAX_H, H_ALIGN, S_ALIGN);
- crop.left = 0;
- crop.top = 0;
+ crop->top = 0;
+ crop->width = sdformat->format.width;
+ crop->height = sdformat->format.height;
+ csi_try_crop(priv, crop, cfg, &sdformat->format, sensor);
- cc = imx_media_find_mbus_format(sdformat->format.code,
- CS_SEL_ANY, true);
- if (!cc) {
+ *cc = imx_media_find_mbus_format(sdformat->format.code,
+ CS_SEL_ANY, true);
+ if (!*cc) {
imx_media_enum_mbus_format(&code, 0,
CS_SEL_ANY, false);
- cc = imx_media_find_mbus_format(code,
+ *cc = imx_media_find_mbus_format(code,
CS_SEL_ANY, false);
- sdformat->format.code = cc->codes[0];
+ sdformat->format.code = (*cc)->codes[0];
}
break;
- default:
- ret = -EINVAL;
+ }
+}
+
+static int csi_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_pix_format vdev_fmt;
+ struct v4l2_rect crop;
+ return -EINVAL;
+
+ sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
}
+ csi_try_fmt(priv, sensor, cfg, sdformat, &crop, &cc);
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
- /* Reset the crop window if this is the input pad */
- if (sdformat->pad == CSI_SINK_PAD)
- priv->crop = crop;
+ goto out;
+ int pad;
+
+ /* reset the crop window */
+ priv->crop = crop;
+
+ /* propagate format to source pads */
+ for (pad = CSI_SINK_PAD + 1; pad < CSI_NUM_PADS; pad++) {
+
+ format.pad = pad;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ csi_try_fmt(priv, sensor, cfg, &format, &crop, &outcc);
+
+ priv->format_mbus[pad] = format.format;
+ priv->cc[pad] = outcc;
+ }
+ }
+
+ /* propagate IDMAC output pad format to capture device */
+ imx_media_mbus_fmt_to_pix_fmt(&vdev_fmt,
+ &priv->format_mbus[CSI_SRC_PAD_IDMAC],
+ priv->cc[CSI_SRC_PAD_IDMAC]);
+ mutex_unlock(&priv->lock);
+ imx_media_capture_device_set_format(vdev, &vdev_fmt);
+
+ return 0;
out:
mutex_unlock(&priv->lock);
return ret;
@@ -1295,9 +1329,7 @@ static int csi_set_selection(struct v4l2_subdev *sd,
}
infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sel->which);
- ret = csi_try_crop(priv, &sel->r, cfg, infmt, sensor);
if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
cfg->try_crop = sel->r;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 070f9da..58eda18 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -562,31 +562,20 @@ static int vdic_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int vdic_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void vdic_try_fmt(struct vdic_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ const struct imx_media_pixfmt **cc)
{
- struct vdic_priv *priv = v4l2_get_subdevdata(sd);
- const struct imx_media_pixfmt *cc;
struct v4l2_mbus_framefmt *infmt;
- int ret = 0;
- u32 code;
-
- if (sdformat->pad >= VDIC_NUM_PADS)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
+ *cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
+ if (!*cc) {
+ u32 code;
- cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
- if (!cc) {
imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
- cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
+ sdformat->format.code = (*cc)->codes[0];
}
switch (sdformat->pad) {
@@ -609,18 +598,51 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
sdformat->format.field = V4L2_FIELD_SEQ_TB;
break;
- default:
- ret = -EINVAL;
+ }
+}
+
}
+ vdic_try_fmt(priv, cfg, sdformat, &cc);
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
+ goto out;
+
+ format.pad = VDIC_SRC_PAD_DIRECT;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ vdic_try_fmt(priv, cfg, &format, &outcc);
+
+ priv->format_mbus[VDIC_SRC_PAD_DIRECT] = format.format;
+ priv->cc[VDIC_SRC_PAD_DIRECT] = outcc;
+ }
out:
mutex_unlock(&priv->lock);
return ret;
--
2.7.4
*_try_fmt(), so that the latter function can be used to propagate
a legal format from sink to source. This also reduces subsequent
bloat in *_set_fmt().
imx-ic-prp never needed separate formats for sink and source pads,
so propagation in this case was easy, just have only a single
format shared by both pads.
drivers/staging/media/imx/imx-ic-prp.c | 31 +++---
drivers/staging/media/imx/imx-ic-prpencvf.c | 86 ++++++++++-----
drivers/staging/media/imx/imx-media-capture.c | 12 ++
drivers/staging/media/imx/imx-media-csi.c | 152 ++++++++++++++++----------
drivers/staging/media/imx/imx-media-vdic.c | 72 +++++++-----
5 files changed, 224 insertions(+), 129 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 83cd2b4..ec742e6 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -56,8 +56,7 @@ struct prp_priv {
/* the CSI id at link validate */
int csi_id;
- struct v4l2_mbus_framefmt format_mbus[PRP_NUM_PADS];
- const struct imx_media_pixfmt *cc[PRP_NUM_PADS];
+ struct v4l2_mbus_framefmt format_mbus;
struct v4l2_fract frame_interval;
bool stream_on; /* streaming is on */
if (which == V4L2_SUBDEV_FORMAT_TRY)
return v4l2_subdev_get_try_format(&ic_priv->sd, cfg, pad);
else
- return &priv->format_mbus[pad];
+ return &priv->format_mbus;
}
/*
@@ -167,7 +166,7 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sdformat)
{
struct prp_priv *priv = sd_to_priv(sd);
- const struct imx_media_pixfmt *cc = NULL;
+ const struct imx_media_pixfmt *cc;
struct v4l2_mbus_framefmt *infmt;
int ret = 0;
u32 code;
@@ -201,17 +200,14 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
/* Output pads mirror input pad */
infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD,
sdformat->which);
- cc = imx_media_find_ipu_format(infmt->code, CS_SEL_ANY);
sdformat->format = *infmt;
break;
}
- if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
cfg->try_fmt = sdformat->format;
- } else {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
- }
+ else
+ priv->format_mbus = sdformat->format;
out:
mutex_unlock(&priv->lock);
@@ -427,20 +423,19 @@ static int prp_registered(struct v4l2_subdev *sd)
for (i = 0; i < PRP_NUM_PADS; i++) {
priv->pad[i].flags = (i == PRP_SINK_PAD) ?
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
-
- /* set a default mbus format */
- imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
- ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
- 640, 480, code, V4L2_FIELD_NONE,
- &priv->cc[i]);
- if (ret)
- return ret;
}
/* init default frame interval */
priv->frame_interval.numerator = 1;
/* init default frame interval */
priv->frame_interval.denominator = 30;
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus, 640, 480, code,
+ V4L2_FIELD_NONE, NULL);
+ if (ret)
+ return ret;
+
+ return ret;
+
return media_entity_pads_init(&sd->entity, PRP_NUM_PADS, priv->pad);
}
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
}
index b42103c..644dd33 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -753,35 +753,23 @@ static int prp_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int prp_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void prp_try_fmt(struct prp_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ const struct imx_media_pixfmt **cc)
{
- struct prp_priv *priv = sd_to_priv(sd);
- const struct imx_media_pixfmt *cc;
- struct v4l2_mbus_framefmt *infmt;
- int ret = 0;
- u32 code;
-
- if (sdformat->pad >= PRPENCVF_NUM_PADS)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
+ *cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
+ if (!*cc) {
+ u32 code;
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
-
- cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
- if (!cc) {
imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
- cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ sdformat->format.code = (*cc)->codes[0];
}
if (sdformat->pad == PRPENCVF_SRC_PAD) {
- infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
if (sdformat->pad == PRPENCVF_SRC_PAD) {
+ struct v4l2_mbus_framefmt *infmt =
+ __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
sdformat->which);
if (sdformat->format.field != V4L2_FIELD_NONE)
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
}
+}
+
+static int prp_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ const struct imx_media_pixfmt *cc;
+ struct v4l2_pix_format vdev_fmt;
+ int ret = 0;
+
+ if (sdformat->pad >= PRPENCVF_NUM_PADS)
+
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ prp_try_fmt(priv, cfg, sdformat, &cc);
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
cfg->try_fmt = sdformat->format;
- } else {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
+ goto out;
+ }
+
+ }
+
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+
+ /* propagate a default format to source pad */
+ priv->cc[sdformat->pad] = cc;
+
+ if (sdformat->pad == PRPENCVF_SINK_PAD) {
+ const struct imx_media_pixfmt *outcc;
+ struct v4l2_subdev_format format;
+
+ format.pad = PRPENCVF_SRC_PAD;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ prp_try_fmt(priv, cfg, &format, &outcc);
+
+ priv->format_mbus[PRPENCVF_SRC_PAD] = format.format;
+ priv->cc[PRPENCVF_SRC_PAD] = outcc;
}
+ /* propagate output pad format to capture device */
+ imx_media_mbus_fmt_to_pix_fmt(&vdev_fmt,
+ &priv->format_mbus[PRPENCVF_SRC_PAD],
+ priv->cc[PRPENCVF_SRC_PAD]);
+ mutex_unlock(&priv->lock);
+ imx_media_capture_device_set_format(vdev, &vdev_fmt);
+
+ return 0;
out:
mutex_unlock(&priv->lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index a757e05..ee91439 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -498,6 +498,18 @@ static struct video_device capture_videodev = {
.device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
};
+void imx_media_capture_device_set_format(struct imx_media_video_dev *vdev,
+ struct v4l2_pix_format *pix)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+
+ mutex_lock(&priv->mutex);
+ priv->vdev.fmt.fmt.pix = *pix;
+ priv->vdev.cc = imx_media_find_format(pix->pixelformat, CS_SEL_ANY);
+ mutex_unlock(&priv->mutex);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_set_format);
+
struct imx_media_buffer *
imx_media_capture_device_next_buf(struct imx_media_video_dev *vdev)
{
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index cf070be..fc0036a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -986,11 +986,11 @@ __csi_get_fmt(struct csi_priv *priv, struct v4l2_subdev_pad_config *cfg,
return &priv->format_mbus[pad];
}
-static int csi_try_crop(struct csi_priv *priv,
- struct v4l2_rect *crop,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_mbus_framefmt *infmt,
- struct imx_media_subdev *sensor)
+static void csi_try_crop(struct csi_priv *priv,
+ struct v4l2_rect *crop,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_mbus_framefmt *infmt,
+ struct imx_media_subdev *sensor)
{
struct v4l2_of_endpoint *sensor_ep;
@@ -1019,8 +1019,6 @@ static int csi_try_crop(struct csi_priv *priv,
if (crop->top + crop->height > infmt->height)
crop->top = infmt->height - crop->height;
}
-
- return 0;
}
static int csi_enum_mbus_code(struct v4l2_subdev *sd,
@@ -1092,34 +1090,17 @@ static int csi_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int csi_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void csi_try_fmt(struct csi_priv *priv,
+ struct imx_media_subdev *sensor,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ struct v4l2_rect *crop,
+ const struct imx_media_pixfmt **cc)
{
- struct csi_priv *priv = v4l2_get_subdevdata(sd);
- const struct imx_media_pixfmt *cc, *incc;
+ const struct imx_media_pixfmt *incc;
struct v4l2_mbus_framefmt *infmt;
- struct imx_media_subdev *sensor;
- struct v4l2_rect crop;
- int ret = 0;
u32 code;
- if (sdformat->pad >= CSI_NUM_PADS)
- return -EINVAL;
-
- sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
- if (IS_ERR(sensor)) {
- v4l2_err(&priv->sd, "no sensor attached\n");
- return PTR_ERR(sensor);
- }
-
- mutex_lock(&priv->lock);
-
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
-
switch (sdformat->pad) {
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
@@ -1140,17 +1121,17 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
if (incc->bayer) {
sdformat->format.code = infmt->code;
- cc = incc;
+ *cc = incc;
} else {
u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
CS_SEL_YUV : CS_SEL_RGB;
- cc = imx_media_find_ipu_format(sdformat->format.code,
- cs_sel);
- if (!cc) {
+ *cc = imx_media_find_ipu_format(sdformat->format.code,
+ cs_sel);
+ if (!*cc) {
imx_media_enum_ipu_format(&code, 0, cs_sel);
- cc = imx_media_find_ipu_format(code, cs_sel);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, cs_sel);
+ sdformat->format.code = (*cc)->codes[0];
}
}
@@ -1172,39 +1153,92 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
W_ALIGN, &sdformat->format.height,
MIN_H, MAX_H, H_ALIGN, S_ALIGN);
- crop.left = 0;
- crop.top = 0;
- crop.width = sdformat->format.width;
- crop.height = sdformat->format.height;
- ret = csi_try_crop(priv, &crop, cfg, &sdformat->format, sensor);
- crop.height = sdformat->format.height;
- if (ret)
- goto out;
+ crop->left = 0;
- goto out;
+ crop->top = 0;
+ crop->width = sdformat->format.width;
+ crop->height = sdformat->format.height;
+ csi_try_crop(priv, crop, cfg, &sdformat->format, sensor);
- cc = imx_media_find_mbus_format(sdformat->format.code,
- CS_SEL_ANY, true);
- if (!cc) {
+ *cc = imx_media_find_mbus_format(sdformat->format.code,
+ CS_SEL_ANY, true);
+ if (!*cc) {
imx_media_enum_mbus_format(&code, 0,
CS_SEL_ANY, false);
- cc = imx_media_find_mbus_format(code,
+ *cc = imx_media_find_mbus_format(code,
CS_SEL_ANY, false);
- sdformat->format.code = cc->codes[0];
+ sdformat->format.code = (*cc)->codes[0];
}
break;
- default:
- ret = -EINVAL;
+ }
+}
+
+static int csi_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ const struct imx_media_pixfmt *cc;
+ struct imx_media_subdev *sensor;
+ struct v4l2_pix_format vdev_fmt;
+ struct v4l2_rect crop;
+ int ret = 0;
+
+ if (sdformat->pad >= CSI_NUM_PADS)
+
+ return -EINVAL;
+
+ sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
goto out;
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
}
+ csi_try_fmt(priv, sensor, cfg, sdformat, &crop, &cc);
+
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
cfg->try_fmt = sdformat->format;
- } else {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
- /* Reset the crop window if this is the input pad */
- if (sdformat->pad == CSI_SINK_PAD)
- priv->crop = crop;
+ goto out;
}
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+
+ if (sdformat->pad == CSI_SINK_PAD) {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+
+ int pad;
+
+ /* reset the crop window */
+ priv->crop = crop;
+
+ /* propagate format to source pads */
+ for (pad = CSI_SINK_PAD + 1; pad < CSI_NUM_PADS; pad++) {
+ const struct imx_media_pixfmt *outcc;
+ struct v4l2_subdev_format format;
+
+ format.pad = pad;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ csi_try_fmt(priv, sensor, cfg, &format, &crop, &outcc);
+
+ priv->format_mbus[pad] = format.format;
+ priv->cc[pad] = outcc;
+ }
+ }
+
+ /* propagate IDMAC output pad format to capture device */
+ imx_media_mbus_fmt_to_pix_fmt(&vdev_fmt,
+ &priv->format_mbus[CSI_SRC_PAD_IDMAC],
+ priv->cc[CSI_SRC_PAD_IDMAC]);
+ mutex_unlock(&priv->lock);
+ imx_media_capture_device_set_format(vdev, &vdev_fmt);
+
+ return 0;
out:
mutex_unlock(&priv->lock);
return ret;
@@ -1295,9 +1329,7 @@ static int csi_set_selection(struct v4l2_subdev *sd,
}
infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sel->which);
- ret = csi_try_crop(priv, &sel->r, cfg, infmt, sensor);
- if (ret)
- goto out;
+ csi_try_crop(priv, &sel->r, cfg, infmt, sensor);
- goto out;
if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
cfg->try_crop = sel->r;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 070f9da..58eda18 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -562,31 +562,20 @@ static int vdic_get_fmt(struct v4l2_subdev *sd,
return ret;
}
-static int vdic_set_fmt(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *sdformat)
+static void vdic_try_fmt(struct vdic_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat,
+ const struct imx_media_pixfmt **cc)
{
- struct vdic_priv *priv = v4l2_get_subdevdata(sd);
- const struct imx_media_pixfmt *cc;
struct v4l2_mbus_framefmt *infmt;
- int ret = 0;
- u32 code;
-
- if (sdformat->pad >= VDIC_NUM_PADS)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
- if (priv->stream_on) {
- ret = -EBUSY;
- goto out;
- }
+ *cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
+ if (!*cc) {
+ u32 code;
- cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
- if (!cc) {
imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
- cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
- sdformat->format.code = cc->codes[0];
+ *cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
+ sdformat->format.code = (*cc)->codes[0];
}
switch (sdformat->pad) {
@@ -609,18 +598,51 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
sdformat->format.field = V4L2_FIELD_SEQ_TB;
break;
- default:
- ret = -EINVAL;
+ }
+}
+
+static int vdic_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ const struct imx_media_pixfmt *cc;
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ const struct imx_media_pixfmt *cc;
+ int ret = 0;
+
+ if (sdformat->pad >= VDIC_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+
+ if (sdformat->pad >= VDIC_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
goto out;
+ ret = -EBUSY;
}
+ vdic_try_fmt(priv, cfg, sdformat, &cc);
+
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
cfg->try_fmt = sdformat->format;
- } else {
- priv->format_mbus[sdformat->pad] = sdformat->format;
- priv->cc[sdformat->pad] = cc;
+ goto out;
}
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+
+ /* propagate format to source pad */
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+
+ if (sdformat->pad == VDIC_SINK_PAD_DIRECT ||
+ sdformat->pad == VDIC_SINK_PAD_IDMAC) {
+ const struct imx_media_pixfmt *outcc;
+ struct v4l2_subdev_format format;
+
+ format.pad = VDIC_SRC_PAD_DIRECT;
+ format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ format.format = sdformat->format;
+ vdic_try_fmt(priv, cfg, &format, &outcc);
+
+ priv->format_mbus[VDIC_SRC_PAD_DIRECT] = format.format;
+ priv->cc[VDIC_SRC_PAD_DIRECT] = outcc;
+ }
out:
mutex_unlock(&priv->lock);
return ret;
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
This is the capture device interface driver that provides the v4l2
user interface. Frames can be received from various sources:
- directly from CSI for capturing unconverted images directly from
camera sensors.
- from the IC pre-process encode task.
- from the IC pre-process viewfinder task.
drivers/staging/media/imx/Makefile | 1 +
drivers/staging/media/imx/imx-media-capture.c | 655 ++++++++++++++++++++++++++
2 files changed, 656 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-capture.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index ddd7d94..4606a3a 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -3,3 +3,4 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
new file mode 100644
index 0000000..704ed85
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -0,0 +1,655 @@
+/*
+ * Video Capture Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2012-2016 Mentor Graphics Inc.
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/pinctrl/consumer.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-of.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+#include <video/imx-ipu-v3.h>
+ struct imx_media_video_dev vdev;
+
+ struct v4l2_subdev *src_sd;
+ int src_sd_pad;
+
+ struct mutex mutex; /* capture device mutex */
+
+ /* the videobuf2 queue */
+ struct vb2_queue q;
+ /* list of ready imx_media_buffer's from q */
+ struct list_head ready_q;
+ /* protect ready_q */
+ spinlock_t q_lock;
+
+ /* controls inherited from subdevs */
+ bool stop; /* streaming is stopping */
+};
+
+#define to_capture_priv(v) container_of(v, struct capture_priv, vdev)
+
+/* In bytes, per queue */
+#define VID_MEM_LIMIT SZ_64M
+
+static struct vb2_ops capture_qops;
+
+/*
+ * Video ioctls follow
+ */
+
+static int vidioc_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ strncpy(cap->driver, "imx-media-capture", sizeof(cap->driver) - 1);
+ strncpy(cap->card, "imx-media-capture", sizeof(cap->card) - 1);
+ snprintf(cap->bus_info, sizeof(cap->bus_info),
+ "platform:%s", dev_name(priv->dev));
+ struct v4l2_fmtdesc *f)
+{
+ u32 fourcc;
+ int ret;
+
+ ret = imx_media_enum_format(&fourcc, NULL, f->index, true, true);
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ *f = priv->vdev.fmt;
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_format fmt_src;
+ const struct imx_media_pixfmt *cc, *src_cc;
+ u32 fourcc;
+ int ret;
+
+ fourcc = f->fmt.pix.pixelformat;
+ cc = imx_media_find_format(fourcc, 0, true, true);
+ if (!cc) {
+ imx_media_enum_format(&fourcc, NULL, 0, true, true);
+ cc = imx_media_find_format(fourcc, 0, true, true);
+ }
+
+ /*
+ * user frame dimensions are the same as src_sd's pad.
+ */
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
+ * but we can allow planar pixel formats if the src_sd's
+ * pad configured a YUV format
+ */
+ src_cc = imx_media_find_format(0, fmt_src.format.code, true, false);
+ if (src_cc->cs == IPUV3_COLORSPACE_YUV &&
+ cc->cs == IPUV3_COLORSPACE_YUV) {
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
+ &fmt_src.format, cc);
+ } else {
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
+ &fmt_src.format, src_cc);
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ int ret;
+
+ if (vb2_is_busy(&priv->q)) {
+ v4l2_err(priv->src_sd, "%s queue busy\n", __func__);
+ return -EBUSY;
+ }
+
+ ret = capture_try_fmt_vid_cap(file, priv, f);
+ priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat, 0,
+ true, true);
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ return v4l2_subdev_call(priv->src_sd, video, querystd, std);
+}
+
+static int capture_g_std(struct file *file, void *fh, v4l2_std_id *std)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ return v4l2_subdev_call(priv->src_sd, video, g_std, std);
+}
+
+static int capture_s_std(struct file *file, void *fh, v4l2_std_id std)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ if (vb2_is_busy(&priv->q))
+ return -EBUSY;
+
+ return v4l2_subdev_call(priv->src_sd, video, s_std, std);
+}
+
+static int capture_g_parm(struct file *file, void *fh,
+ struct v4l2_streamparm *a)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_frame_interval fi;
+ int ret;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ memset(&fi, 0, sizeof(fi));
+ fi.pad = priv->src_sd_pad;
+ ret = v4l2_subdev_call(priv->src_sd, video, g_frame_interval, &fi);
+ if (ret < 0)
+ return ret;
+
+ a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ a->parm.capture.timeperframe = fi.interval;
+ struct v4l2_streamparm *a)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_frame_interval fi;
+ int ret;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ memset(&fi, 0, sizeof(fi));
+ fi.pad = priv->src_sd_pad;
+ fi.interval = a->parm.capture.timeperframe;
+ ret = v4l2_subdev_call(priv->src_sd, video, s_frame_interval, &fi);
+ if (ret < 0)
+ return ret;
+
+ a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ a->parm.capture.timeperframe = fi.interval;
+ .vidioc_querycap = vidioc_querycap,
+
+ .vidioc_enum_fmt_vid_cap = capture_enum_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap = capture_g_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = capture_try_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = capture_s_fmt_vid_cap,
+
+ .vidioc_querystd = capture_querystd,
+ .vidioc_g_std = capture_g_std,
+ .vidioc_s_std = capture_s_std,
+
+ .vidioc_g_parm = capture_g_parm,
+ .vidioc_s_parm = capture_s_parm,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+};
+
+/*
+ * Queue operations
+ */
+
+static int capture_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format *pix = &priv->vdev.fmt.fmt.pix;
+ unsigned int count = *nbuffers;
+
+ if (vq->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ if (*nplanes) {
+ if (*nplanes != 1 || sizes[0] < pix->sizeimage)
+ return -EINVAL;
+ count += vq->num_buffers;
+ }
+
+ count = min_t(__u32, VID_MEM_LIMIT / pix->sizeimage, count);
+
+ if (*nplanes)
+ *nbuffers = (count < vq->num_buffers) ? 0 :
+ count - vq->num_buffers;
+ else
+ *nbuffers = count;
+
+ *nplanes = 1;
+ sizes[0] = pix->sizeimage;
+{
+ struct imx_media_buffer *buf = to_imx_media_vb(vb);
+
+ INIT_LIST_HEAD(&buf->list);
+{
+ struct vb2_queue *vq = vb->vb2_queue;
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format *pix = &priv->vdev.fmt.fmt.pix;
+
+ if (vb2_plane_size(vb, 0) < pix->sizeimage) {
+ v4l2_err(priv->src_sd,
+ "data will not fit into plane (%lu < %lu)\n",
+ vb2_plane_size(vb, 0), (long)pix->sizeimage);
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vb->vb2_queue);
+ struct imx_media_buffer *buf = to_imx_media_vb(vb);
+
+ list_add_tail(&buf->list, &priv->ready_q);
+
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+
+static int capture_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct imx_media_buffer *buf, *tmp;
+ unsigned long flags;
+ int ret;
+
+ if (vb2_is_streaming(vq))
+ return 0;
+
+ ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
+ &priv->mp, true);
+ if (ret) {
+ v4l2_err(priv->src_sd, "pipeline start failed with %d\n", ret);
+ goto return_bufs;
+ }
+
+ priv->stop = false;
+ spin_lock_irqsave(&priv->q_lock, flags);
+ list_for_each_entry_safe(buf, tmp, &priv->ready_q, list) {
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vbuf.vb2_buf, VB2_BUF_STATE_QUEUED);
+ }
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct imx_media_buffer *frame;
+ unsigned long flags;
+ int ret;
+
+ if (!vb2_is_streaming(vq))
+ return;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+ priv->stop = true;
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+
+ ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
+ &priv->mp, false);
+ if (ret)
+ v4l2_warn(priv->src_sd, "pipeline stop failed with %d\n", ret);
+
+ /* release all active buffers */
+ spin_lock_irqsave(&priv->q_lock, flags);
+ while (!list_empty(&priv->ready_q)) {
+ frame = list_entry(priv->ready_q.next,
+ struct imx_media_buffer, list);
+ list_del(&frame->list);
+ vb2_buffer_done(&frame->vbuf.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+
+static struct vb2_ops capture_qops = {
+ .queue_setup = capture_queue_setup,
+ .buf_init = capture_buf_init,
+ .buf_prepare = capture_buf_prepare,
+ .buf_queue = capture_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = capture_start_streaming,
+ .stop_streaming = capture_stop_streaming,
+};
+
+/*
+ * File operations
+ */
+static int capture_open(struct file *file)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ int ret;
+
+ if (mutex_lock_interruptible(&priv->mutex))
+ return -ERESTARTSYS;
+
+ ret = v4l2_fh_open(file);
+ if (ret)
+ v4l2_err(priv->src_sd, "v4l2_fh_open failed\n");
+
+ mutex_unlock(&priv->mutex);
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct vb2_queue *vq = &priv->q;
+
+ if (file->private_data == vq->owner) {
+ vb2_queue_release(vq);
+ vq->owner = NULL;
+ }
+
+ v4l2_fh_release(file);
+ mutex_unlock(&priv->mutex);
+ .owner = THIS_MODULE,
+ .open = capture_open,
+ .release = capture_release,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap,
+};
+
+static struct video_device capture_videodev = {
+ .fops = &capture_fops,
+ .ioctl_ops = &capture_ioctl_ops,
+ .minor = -1,
+ .release = video_device_release,
+ .vfl_dir = VFL_DIR_RX,
+ .tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL | V4L2_STD_SECAM,
+ .device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
+};
+
+struct imx_media_buffer *
+imx_media_capture_device_next_buf(struct imx_media_video_dev *vdev)
+ buf = list_entry(priv->ready_q.next, struct imx_media_buffer,
+ list);
+ list_del(&buf->list);
+ }
+
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+
+ return buf;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_next_buf);
+
+void imx_media_capture_device_error(struct imx_media_video_dev *vdev)
+ return;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+ vb2_queue_error(vq);
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_error);
+
+int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
+ struct video_device *vfd = vdev->vfd;
+ struct vb2_queue *vq = &priv->q;
+ struct v4l2_subdev_format fmt_src;
+ int ret;
+
+
+ ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
+ if (ret) {
+ v4l2_err(sd, "Failed to register video device\n");
+ vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ vq->drv_priv = priv;
+ vq->buf_struct_size = sizeof(struct imx_media_buffer);
+ vq->ops = &capture_qops;
+ vq->mem_ops = &vb2_dma_contig_memops;
+ vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vq->lock = &priv->mutex;
+ vq->min_buffers_needed = 2;
+ vq->dev = priv->dev;
+
+ ret = vb2_queue_init(vq);
+ if (ret) {
+ v4l2_err(sd, "vb2_queue_init failed\n");
+ goto unreg;
+ }
+
+ INIT_LIST_HEAD(&priv->ready_q);
+
+ priv->vdev_pad.flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&vfd->entity, 1, &priv->vdev_pad);
+ if (ret) {
+ v4l2_err(sd, "failed to init dev pad\n");
+ goto unreg;
+ }
+
+ /* create the link from the src_sd devnode pad to device node */
+ ret = media_create_pad_link(&sd->entity, priv->src_sd_pad,
+ &vfd->entity, 0, 0);
+ if (ret) {
+ v4l2_err(sd, "failed to create link to device node\n");
+ goto unreg;
+ }
+
+ /* setup default format */
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt_src);
+ if (ret) {
+ v4l2_err(sd, "failed to get src_sd format\n");
+ goto unreg;
+ }
+
+ vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt.fmt.pix,
+ &fmt_src.format, NULL);
+ vdev->cc = imx_media_find_format(0, fmt_src.format.code,
+ true, false);
+
+ v4l2_info(sd, "Registered %s as /dev/%s\n", vfd->name,
+ video_device_node_name(vfd));
+
+ vfd->ctrl_handler = &priv->ctrl_hdlr;
+
+ return 0;
+unreg:
+ video_unregister_device(vfd);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_register);
+
+void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev)
+ video_unregister_device(vfd);
+ media_entity_cleanup(&vfd->entity);
+ }
+
+ mutex_unlock(&priv->mutex);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_unregister);
+
+struct imx_media_video_dev *
+imx_media_capture_device_init(struct v4l2_subdev *src_sd, int pad)
+{
+ struct capture_priv *priv;
+ struct video_device *vfd;
+
+ priv = devm_kzalloc(src_sd->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return ERR_PTR(-ENOMEM);
+
+ priv->src_sd = src_sd;
+ priv->src_sd_pad = pad;
+ priv->dev = src_sd->dev;
+
+ mutex_init(&priv->mutex);
+ spin_lock_init(&priv->q_lock);
+
+ snprintf(capture_videodev.name, sizeof(capture_videodev.name),
+ "%s capture", src_sd->name);
+
+ vfd = video_device_alloc();
+ if (!vfd)
+ return ERR_PTR(-ENOMEM);
+
+ *vfd = capture_videodev;
+ vfd->lock = &priv->mutex;
+ vfd->queue = &priv->q;
+ priv->vdev.vfd = vfd;
+
+ video_set_drvdata(vfd, priv);
+
+ v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
+
+ return &priv->vdev;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_init);
+
+void imx_media_capture_device_remove(struct imx_media_video_dev *vdev)
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_remove);
+
+MODULE_DESCRIPTION("i.MX5/6 v4l2 video capture interface driver");
--
2.7.4
user interface. Frames can be received from various sources:
- directly from CSI for capturing unconverted images directly from
camera sensors.
- from the IC pre-process encode task.
- from the IC pre-process viewfinder task.
drivers/staging/media/imx/Makefile | 1 +
drivers/staging/media/imx/imx-media-capture.c | 655 ++++++++++++++++++++++++++
2 files changed, 656 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-capture.c
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index ddd7d94..4606a3a 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -3,3 +3,4 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
new file mode 100644
index 0000000..704ed85
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -0,0 +1,655 @@
+/*
+ * Video Capture Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2012-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/fs.h>
+ * (at your option) any later version.
+ */
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-of.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+#include <video/imx-ipu-v3.h>
+#include <media/imx.h>
+#include "imx-media.h"
+
+struct capture_priv {
+#include "imx-media.h"
+
+ struct imx_media_video_dev vdev;
+
+ struct v4l2_subdev *src_sd;
+ int src_sd_pad;
+ struct device *dev;
+
+ struct media_pipeline mp;
+
+ struct imx_media_dev *md;
+
+ struct media_pad vdev_pad;
+
+
+ struct mutex mutex; /* capture device mutex */
+
+ /* the videobuf2 queue */
+ struct vb2_queue q;
+ /* list of ready imx_media_buffer's from q */
+ struct list_head ready_q;
+ /* protect ready_q */
+ spinlock_t q_lock;
+
+ /* controls inherited from subdevs */
+ struct v4l2_ctrl_handler ctrl_hdlr;
+
+ /* misc status */
+
+ bool stop; /* streaming is stopping */
+};
+
+#define to_capture_priv(v) container_of(v, struct capture_priv, vdev)
+
+/* In bytes, per queue */
+#define VID_MEM_LIMIT SZ_64M
+
+static struct vb2_ops capture_qops;
+
+/*
+ * Video ioctls follow
+ */
+
+static int vidioc_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ strncpy(cap->driver, "imx-media-capture", sizeof(cap->driver) - 1);
+ strncpy(cap->card, "imx-media-capture", sizeof(cap->card) - 1);
+ snprintf(cap->bus_info, sizeof(cap->bus_info),
+ "platform:%s", dev_name(priv->dev));
+
+ return 0;
+}
+
+static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
+ return 0;
+}
+
+ struct v4l2_fmtdesc *f)
+{
+ u32 fourcc;
+ int ret;
+
+ ret = imx_media_enum_format(&fourcc, NULL, f->index, true, true);
+ if (ret)
+ return ret;
+
+ f->pixelformat = fourcc;
+ return ret;
+
+
+ return 0;
+}
+
+static int capture_g_fmt_vid_cap(struct file *file, void *fh,
+ return 0;
+}
+
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ *f = priv->vdev.fmt;
+
+ return 0;
+}
+
+static int capture_try_fmt_vid_cap(struct file *file, void *fh,
+ return 0;
+}
+
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_format fmt_src;
+ const struct imx_media_pixfmt *cc, *src_cc;
+ u32 fourcc;
+ int ret;
+
+ fourcc = f->fmt.pix.pixelformat;
+ cc = imx_media_find_format(fourcc, 0, true, true);
+ if (!cc) {
+ imx_media_enum_format(&fourcc, NULL, 0, true, true);
+ cc = imx_media_find_format(fourcc, 0, true, true);
+ }
+
+ /*
+ * user frame dimensions are the same as src_sd's pad.
+ */
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
+ if (ret)
+ return ret;
+
+ /*
+ return ret;
+
+ * but we can allow planar pixel formats if the src_sd's
+ * pad configured a YUV format
+ */
+ src_cc = imx_media_find_format(0, fmt_src.format.code, true, false);
+ if (src_cc->cs == IPUV3_COLORSPACE_YUV &&
+ cc->cs == IPUV3_COLORSPACE_YUV) {
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
+ &fmt_src.format, cc);
+ } else {
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
+ &fmt_src.format, src_cc);
+ }
+
+ return 0;
+}
+
+static int capture_s_fmt_vid_cap(struct file *file, void *fh,
+
+ return 0;
+}
+
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ int ret;
+
+ if (vb2_is_busy(&priv->q)) {
+ v4l2_err(priv->src_sd, "%s queue busy\n", __func__);
+ return -EBUSY;
+ }
+
+ ret = capture_try_fmt_vid_cap(file, priv, f);
+ if (ret)
+ return ret;
+
+ priv->vdev.fmt.fmt.pix = f->fmt.pix;
+ return ret;
+
+ priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat, 0,
+ true, true);
+
+ return 0;
+}
+
+static int capture_querystd(struct file *file, void *fh, v4l2_std_id *std)
+ return 0;
+}
+
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ return v4l2_subdev_call(priv->src_sd, video, querystd, std);
+}
+
+static int capture_g_std(struct file *file, void *fh, v4l2_std_id *std)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ return v4l2_subdev_call(priv->src_sd, video, g_std, std);
+}
+
+static int capture_s_std(struct file *file, void *fh, v4l2_std_id std)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ if (vb2_is_busy(&priv->q))
+ return -EBUSY;
+
+ return v4l2_subdev_call(priv->src_sd, video, s_std, std);
+}
+
+static int capture_g_parm(struct file *file, void *fh,
+ struct v4l2_streamparm *a)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_frame_interval fi;
+ int ret;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ memset(&fi, 0, sizeof(fi));
+ fi.pad = priv->src_sd_pad;
+ ret = v4l2_subdev_call(priv->src_sd, video, g_frame_interval, &fi);
+ if (ret < 0)
+ return ret;
+
+ a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ a->parm.capture.timeperframe = fi.interval;
+
+ return 0;
+}
+
+static int capture_s_parm(struct file *file, void *fh,
+ return 0;
+}
+
+ struct v4l2_streamparm *a)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_frame_interval fi;
+ int ret;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ memset(&fi, 0, sizeof(fi));
+ fi.pad = priv->src_sd_pad;
+ fi.interval = a->parm.capture.timeperframe;
+ ret = v4l2_subdev_call(priv->src_sd, video, s_frame_interval, &fi);
+ if (ret < 0)
+ return ret;
+
+ a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ a->parm.capture.timeperframe = fi.interval;
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops capture_ioctl_ops = {
+ return 0;
+}
+
+ .vidioc_querycap = vidioc_querycap,
+
+ .vidioc_enum_fmt_vid_cap = capture_enum_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap = capture_g_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = capture_try_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = capture_s_fmt_vid_cap,
+
+ .vidioc_querystd = capture_querystd,
+ .vidioc_g_std = capture_g_std,
+ .vidioc_s_std = capture_s_std,
+
+ .vidioc_g_parm = capture_g_parm,
+ .vidioc_s_parm = capture_s_parm,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+};
+
+/*
+ * Queue operations
+ */
+
+static int capture_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format *pix = &priv->vdev.fmt.fmt.pix;
+ unsigned int count = *nbuffers;
+
+ if (vq->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ if (*nplanes) {
+ if (*nplanes != 1 || sizes[0] < pix->sizeimage)
+ return -EINVAL;
+ count += vq->num_buffers;
+ }
+
+ count = min_t(__u32, VID_MEM_LIMIT / pix->sizeimage, count);
+
+ if (*nplanes)
+ *nbuffers = (count < vq->num_buffers) ? 0 :
+ count - vq->num_buffers;
+ else
+ *nbuffers = count;
+
+ *nplanes = 1;
+ sizes[0] = pix->sizeimage;
+
+ return 0;
+}
+
+static int capture_buf_init(struct vb2_buffer *vb)
+ return 0;
+}
+
+{
+ struct imx_media_buffer *buf = to_imx_media_vb(vb);
+
+ INIT_LIST_HEAD(&buf->list);
+
+ return 0;
+}
+
+static int capture_buf_prepare(struct vb2_buffer *vb)
+ return 0;
+}
+
+{
+ struct vb2_queue *vq = vb->vb2_queue;
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format *pix = &priv->vdev.fmt.fmt.pix;
+
+ if (vb2_plane_size(vb, 0) < pix->sizeimage) {
+ v4l2_err(priv->src_sd,
+ "data will not fit into plane (%lu < %lu)\n",
+ vb2_plane_size(vb, 0), (long)pix->sizeimage);
+ return -EINVAL;
+ }
+
+ vb2_set_plane_payload(vb, 0, pix->sizeimage);
+ }
+
+
+ return 0;
+}
+
+static void capture_buf_queue(struct vb2_buffer *vb)
+ return 0;
+}
+
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vb->vb2_queue);
+ struct imx_media_buffer *buf = to_imx_media_vb(vb);
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+
+
+ list_add_tail(&buf->list, &priv->ready_q);
+
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+
+static int capture_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct imx_media_buffer *buf, *tmp;
+ unsigned long flags;
+ int ret;
+
+ if (vb2_is_streaming(vq))
+ return 0;
+
+ ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
+ &priv->mp, true);
+ if (ret) {
+ v4l2_err(priv->src_sd, "pipeline start failed with %d\n", ret);
+ goto return_bufs;
+ }
+
+ priv->stop = false;
+
+ return 0;
+
+return_bufs:
+ return 0;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+ list_for_each_entry_safe(buf, tmp, &priv->ready_q, list) {
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vbuf.vb2_buf, VB2_BUF_STATE_QUEUED);
+ }
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+ return ret;
+}
+
+static void capture_stop_streaming(struct vb2_queue *vq)
+}
+
+{
+ struct capture_priv *priv = vb2_get_drv_priv(vq);
+ struct imx_media_buffer *frame;
+ unsigned long flags;
+ int ret;
+
+ if (!vb2_is_streaming(vq))
+ return;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+ priv->stop = true;
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+
+ ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
+ &priv->mp, false);
+ if (ret)
+ v4l2_warn(priv->src_sd, "pipeline stop failed with %d\n", ret);
+
+ /* release all active buffers */
+ spin_lock_irqsave(&priv->q_lock, flags);
+ while (!list_empty(&priv->ready_q)) {
+ frame = list_entry(priv->ready_q.next,
+ struct imx_media_buffer, list);
+ list_del(&frame->list);
+ vb2_buffer_done(&frame->vbuf.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+
+static struct vb2_ops capture_qops = {
+ .queue_setup = capture_queue_setup,
+ .buf_init = capture_buf_init,
+ .buf_prepare = capture_buf_prepare,
+ .buf_queue = capture_buf_queue,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .start_streaming = capture_start_streaming,
+ .stop_streaming = capture_stop_streaming,
+};
+
+/*
+ * File operations
+ */
+static int capture_open(struct file *file)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ int ret;
+
+ if (mutex_lock_interruptible(&priv->mutex))
+ return -ERESTARTSYS;
+
+ ret = v4l2_fh_open(file);
+ if (ret)
+ v4l2_err(priv->src_sd, "v4l2_fh_open failed\n");
+
+ mutex_unlock(&priv->mutex);
+ return ret;
+}
+
+static int capture_release(struct file *file)
+}
+
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct vb2_queue *vq = &priv->q;
+ int ret = 0;
+
+ mutex_lock(&priv->mutex);
+
+
+ if (file->private_data == vq->owner) {
+ vb2_queue_release(vq);
+ vq->owner = NULL;
+ }
+
+ v4l2_fh_release(file);
+ mutex_unlock(&priv->mutex);
+ return ret;
+}
+
+static const struct v4l2_file_operations capture_fops = {
+}
+
+ .owner = THIS_MODULE,
+ .open = capture_open,
+ .release = capture_release,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap,
+};
+
+static struct video_device capture_videodev = {
+ .fops = &capture_fops,
+ .ioctl_ops = &capture_ioctl_ops,
+ .minor = -1,
+ .release = video_device_release,
+ .vfl_dir = VFL_DIR_RX,
+ .tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL | V4L2_STD_SECAM,
+ .device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
+};
+
+struct imx_media_buffer *
+imx_media_capture_device_next_buf(struct imx_media_video_dev *vdev)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+ struct imx_media_buffer *buf = NULL;
+ struct capture_priv *priv = to_capture_priv(vdev);
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+
+
+ /* get next queued buffer */
+ if (!list_empty(&priv->ready_q)) {
+ /* get next queued buffer */
+ buf = list_entry(priv->ready_q.next, struct imx_media_buffer,
+ list);
+ list_del(&buf->list);
+ }
+
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+
+ return buf;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_next_buf);
+
+void imx_media_capture_device_error(struct imx_media_video_dev *vdev)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+ struct vb2_queue *vq = &priv->q;
+ struct capture_priv *priv = to_capture_priv(vdev);
+ unsigned long flags;
+
+ if (!vb2_is_streaming(vq))
+
+ return;
+
+ spin_lock_irqsave(&priv->q_lock, flags);
+ vb2_queue_error(vq);
+ spin_unlock_irqrestore(&priv->q_lock, flags);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_error);
+
+int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+ struct v4l2_subdev *sd = priv->src_sd;
+ struct capture_priv *priv = to_capture_priv(vdev);
+ struct video_device *vfd = vdev->vfd;
+ struct vb2_queue *vq = &priv->q;
+ struct v4l2_subdev_format fmt_src;
+ int ret;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ vfd->v4l2_dev = sd->v4l2_dev;
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+
+ ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
+ if (ret) {
+ v4l2_err(sd, "Failed to register video device\n");
+ return ret;
+ }
+
+ vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ }
+
+ vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ vq->drv_priv = priv;
+ vq->buf_struct_size = sizeof(struct imx_media_buffer);
+ vq->ops = &capture_qops;
+ vq->mem_ops = &vb2_dma_contig_memops;
+ vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vq->lock = &priv->mutex;
+ vq->min_buffers_needed = 2;
+ vq->dev = priv->dev;
+
+ ret = vb2_queue_init(vq);
+ if (ret) {
+ v4l2_err(sd, "vb2_queue_init failed\n");
+ goto unreg;
+ }
+
+ INIT_LIST_HEAD(&priv->ready_q);
+
+ priv->vdev_pad.flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&vfd->entity, 1, &priv->vdev_pad);
+ if (ret) {
+ v4l2_err(sd, "failed to init dev pad\n");
+ goto unreg;
+ }
+
+ /* create the link from the src_sd devnode pad to device node */
+ ret = media_create_pad_link(&sd->entity, priv->src_sd_pad,
+ &vfd->entity, 0, 0);
+ if (ret) {
+ v4l2_err(sd, "failed to create link to device node\n");
+ goto unreg;
+ }
+
+ /* setup default format */
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt_src);
+ if (ret) {
+ v4l2_err(sd, "failed to get src_sd format\n");
+ goto unreg;
+ }
+
+ vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt.fmt.pix,
+ &fmt_src.format, NULL);
+ vdev->cc = imx_media_find_format(0, fmt_src.format.code,
+ true, false);
+
+ v4l2_info(sd, "Registered %s as /dev/%s\n", vfd->name,
+ video_device_node_name(vfd));
+
+ vfd->ctrl_handler = &priv->ctrl_hdlr;
+
+ return 0;
+unreg:
+ video_unregister_device(vfd);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_register);
+
+void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+ struct video_device *vfd = priv->vdev.vfd;
+ struct capture_priv *priv = to_capture_priv(vdev);
+
+ mutex_lock(&priv->mutex);
+
+ if (video_is_registered(vfd)) {
+ mutex_lock(&priv->mutex);
+
+ video_unregister_device(vfd);
+ media_entity_cleanup(&vfd->entity);
+ }
+
+ mutex_unlock(&priv->mutex);
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_unregister);
+
+struct imx_media_video_dev *
+imx_media_capture_device_init(struct v4l2_subdev *src_sd, int pad)
+{
+ struct capture_priv *priv;
+ struct video_device *vfd;
+
+ priv = devm_kzalloc(src_sd->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return ERR_PTR(-ENOMEM);
+
+ priv->src_sd = src_sd;
+ priv->src_sd_pad = pad;
+ priv->dev = src_sd->dev;
+
+ mutex_init(&priv->mutex);
+ spin_lock_init(&priv->q_lock);
+
+ snprintf(capture_videodev.name, sizeof(capture_videodev.name),
+ "%s capture", src_sd->name);
+
+ vfd = video_device_alloc();
+ if (!vfd)
+ return ERR_PTR(-ENOMEM);
+
+ *vfd = capture_videodev;
+ vfd->lock = &priv->mutex;
+ vfd->queue = &priv->q;
+ priv->vdev.vfd = vfd;
+
+ video_set_drvdata(vfd, priv);
+
+ v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
+
+ return &priv->vdev;
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_init);
+
+void imx_media_capture_device_remove(struct imx_media_video_dev *vdev)
+{
+ struct capture_priv *priv = to_capture_priv(vdev);
+
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+ struct capture_priv *priv = to_capture_priv(vdev);
+
+}
+EXPORT_SYMBOL_GPL(imx_media_capture_device_remove);
+
+MODULE_DESCRIPTION("i.MX5/6 v4l2 video capture interface driver");
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:05 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
This driver can handle SoC internal and external video bus multiplexers,
controlled either by register bit fields or by a GPIO. The subdevice
passes through frame interval and mbus configuration of the active input
to the output side.
Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
- fixed a cut&paste error in vidsw_remove(): v4l2_async_register_subdev()
should be unregister.
- added media_entity_cleanup() to vidsw_remove().
- added missing MODULE_DEVICE_TABLE().
Suggested-by: Javier Martinez Canillas <jav...@dowhile0.org>
- there was a line left over from a previous iteration that negated
the new way of determining the pad count just before it which
has been removed (num_pads = of_get_child_count(np)).
- enforce that output frame interval must mirror input frame interval
in vidsw_s_frame_interval().
- init frame interval to a default 30 fps.
- add link_validate to media_entity_operations and v4l2_subdev_pad_ops.
- moved devicetree binding doc to a separate commit.
- Philipp Zabel has developed a set of patches that allow adding
to the subdev async notifier waiting list using a chaining method
from the async registered callbacks (v4l2_of_subdev_registered()
and the prep patches for that). For now, I've removed the use of
v4l2_of_subdev_registered() for the vidmux driver's registered
callback. This doesn't affect the functionality of this driver,
but allows for it to be merged now, before adding the chaining
support.
3 files changed, 508 insertions(+)
create mode 100644 drivers/media/platform/video-multiplexer.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 53f6f12..3226eb6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -74,6 +74,14 @@ config VIDEO_M32R_AR_M64278
To compile this driver as a module, choose M here: the
module will be called arv.
+config VIDEO_MULTIPLEXER
+ tristate "Video Multiplexer"
+ depends on VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
+ help
+ This driver provides support for SoC internal N:1 video bus
+ multiplexers controlled by register bitfields as well as external
+ 2:1 video multiplexers controlled by a single GPIO.
+
config VIDEO_OMAP3
tristate "OMAP 3 Camera support"
depends on VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API && ARCH_OMAP3
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 8959f6e..d418add 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -27,6 +27,8 @@ obj-$(CONFIG_VIDEO_SH_VEU) += sh_veu.o
obj-$(CONFIG_VIDEO_MEM2MEM_DEINTERLACE) += m2m-deinterlace.o
+obj-$(CONFIG_VIDEO_MULTIPLEXER) += video-multiplexer.o
+
obj-$(CONFIG_VIDEO_S3C_CAMIF) += s3c-camif/
obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS4_IS) += exynos4-is/
obj-$(CONFIG_VIDEO_SAMSUNG_S5P_JPEG) += s5p-jpeg/
diff --git a/drivers/media/platform/video-multiplexer.c b/drivers/media/platform/video-multiplexer.c
new file mode 100644
index 0000000..02a21072
--- /dev/null
+++ b/drivers/media/platform/video-multiplexer.c
@@ -0,0 +1,498 @@
+/*
+ * video stream multiplexer controlled via gpio or syscon
+ *
+ * Copyright (C) 2013 Pengutronix, Sascha Hauer <ker...@pengutronix.de>
+ * Copyright (C) 2016 Pengutronix, Philipp Zabel <ker...@pengutronix.de>
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/err.h>
+#include <linux/gpio/consumer.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-of.h>
+
+struct vidsw {
+ struct v4l2_subdev subdev;
+ unsigned int num_pads;
+ struct media_pad *pads;
+ struct v4l2_mbus_framefmt *format_mbus;
+ struct v4l2_fract timeperframe;
+ struct v4l2_of_endpoint *endpoint;
+ struct regmap_field *field;
+ struct gpio_desc *gpio;
+ int active;
+};
+
+static inline struct vidsw *v4l2_subdev_to_vidsw(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct vidsw, subdev);
+}
+
+static void vidsw_set_active(struct vidsw *vidsw, int active)
+{
+ vidsw->active = active;
+ if (active < 0)
+ return;
+
+ dev_dbg(vidsw->subdev.dev, "setting %d active\n", active);
+
+ if (vidsw->field)
+ regmap_field_write(vidsw->field, active);
+ else if (vidsw->gpio)
+ gpiod_set_value(vidsw->gpio, active);
+}
+
+static int vidsw_link_setup(struct media_entity *entity,
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+
+ /* We have no limitations on enabling or disabling our output link */
+ if (local->index == vidsw->num_pads - 1)
+ return 0;
+
+ dev_dbg(sd->dev, "link setup %s -> %s", remote->entity->name,
+ if (local->index == vidsw->active) {
+ dev_dbg(sd->dev, "going inactive\n");
+ vidsw->active = -1;
+ struct media_pad *pad;
+
+ if (vidsw->active == local->index)
+ return 0;
+
+ pad = media_entity_remote_pad(&vidsw->pads[vidsw->active]);
+ if (pad) {
+ struct media_link *link;
+ int ret;
+
+ link = media_entity_find_link(pad,
+ &vidsw->pads[vidsw->active]);
+ if (link) {
+ ret = __media_entity_setup_link(link, 0);
+ if (ret)
+ vidsw_set_active(vidsw, local->index);
+ .link_setup = vidsw_link_setup,
+{
+ struct device_node *rpp;
+
+ if (!of_device_is_available(ep))
+ return true;
+
+ rpp = of_graph_get_remote_port_parent(ep);
+ if (!rpp)
+ return true;
+
+ return !of_device_is_available(rpp);
+}
+
+static int vidsw_async_init(struct vidsw *vidsw, struct device_node *node)
+{
+ struct device_node *ep;
+ u32 portno;
+ int numports;
+ int ret;
+ int i;
+ bool active_link = false;
+
+ numports = vidsw->num_pads;
+
+ for (i = 0; i < numports - 1; i++)
+ vidsw->pads[i].flags = MEDIA_PAD_FL_SINK;
+ vidsw->pads[numports - 1].flags = MEDIA_PAD_FL_SOURCE;
+
+ vidsw->subdev.entity.function = MEDIA_ENT_F_VID_MUX;
+ ret = media_entity_pads_init(&vidsw->subdev.entity, numports,
+ vidsw->pads);
+
+ for_each_endpoint_of_node(node, ep) {
+ struct v4l2_of_endpoint endpoint;
+
+ v4l2_of_parse_endpoint(ep, &endpoint);
+
+ portno = endpoint.base.port;
+ if (portno >= numports - 1)
+ continue;
+
+ if (vidsw_endpoint_disabled(ep)) {
+ dev_dbg(vidsw->subdev.dev,
+ "port %d disabled\n", portno);
+ continue;
+ }
+
+ vidsw->endpoint[portno] = endpoint;
+
+ if (portno == vidsw->active)
+ active_link = true;
+ }
+
+ for (portno = 0; portno < numports - 1; portno++) {
+ if (!vidsw->endpoint[portno].base.local_node)
+ continue;
+
+ /* If the active input is not connected, use another */
+ if (!active_link) {
+ vidsw_set_active(vidsw, portno);
+ active_link = true;
+ }
+ }
+
+ return v4l2_async_register_subdev(&vidsw->subdev);
+}
+
+int vidsw_g_mbus_config(struct v4l2_subdev *sd, struct v4l2_mbus_config *cfg)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+ struct media_pad *pad;
+ int ret;
+
+ if (vidsw->active == -1) {
+ dev_err(sd->dev, "no configuration for inactive mux\n");
+ * Retrieve media bus configuration from the entity connected to the
+ * active input
+ */
+ pad = media_entity_remote_pad(&vidsw->pads[vidsw->active]);
+ if (pad) {
+ sd = media_entity_to_v4l2_subdev(pad->entity);
+ ret = v4l2_subdev_call(sd, video, g_mbus_config, cfg);
+ if (ret == -ENOIOCTLCMD)
+ pad = NULL;
+ else if (ret < 0) {
+ dev_err(sd->dev, "failed to get source configuration\n");
+ /* Mirror the input side on the output side */
+ cfg->type = vidsw->endpoint[vidsw->active].bus_type;
+ if (cfg->type == V4L2_MBUS_PARALLEL ||
+ cfg->type == V4L2_MBUS_BT656)
+ cfg->flags = vidsw->endpoint[vidsw->active].bus.parallel.flags;
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+ struct v4l2_subdev *upstream_sd;
+ struct media_pad *pad;
+
+ if (vidsw->active == -1) {
+ dev_err(sd->dev, "Can not start streaming on inactive mux\n");
+ if (!pad) {
+ dev_err(sd->dev, "Failed to find remote source pad\n");
+ return -ENOLINK;
+ }
+
+ if (!is_media_entity_v4l2_subdev(pad->entity)) {
+ dev_err(sd->dev, "Upstream entity is not a v4l2 subdev\n");
+
+ return v4l2_subdev_call(upstream_sd, video, s_stream, enable);
+}
+
+static int vidsw_g_frame_interval(struct v4l2_subdev *sd,
+
+ fi->interval = vidsw->timeperframe;
+
+ /* Output pad mirrors active input pad, no limits on input pads */
+ if (fi->pad == (vidsw->num_pads - 1))
+ fi->interval = vidsw->timeperframe;
+
+ vidsw->timeperframe = fi->interval;
+ .g_mbus_config = vidsw_g_mbus_config,
+ .s_stream = vidsw_s_stream,
+ .g_frame_interval = vidsw_g_frame_interval,
+ .s_frame_interval = vidsw_s_frame_interval,
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, u32 which)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ return v4l2_subdev_get_try_format(sd, cfg, pad);
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ return &vidsw->format_mbus[pad];
+ default:
+ return NULL;
+ }
+}
+
+static int vidsw_get_format(struct v4l2_subdev *sd,
+ sdformat->which);
+ struct v4l2_mbus_framefmt *mbusformat;
+
+ if (sdformat->pad >= vidsw->num_pads)
+ return -EINVAL;
+
+ mbusformat = __vidsw_get_pad_format(sd, cfg, sdformat->pad,
+ sdformat->which);
+ if (!mbusformat)
+ return -EINVAL;
+
+ /* Output pad mirrors active input pad, no limitations on input pads */
+ if (sdformat->pad == (vidsw->num_pads - 1) && vidsw->active >= 0)
+ sdformat->format = vidsw->format_mbus[vidsw->active];
+
+ *mbusformat = sdformat->format;
+
+}
+
+static struct v4l2_subdev_pad_ops vidsw_pad_ops = {
+ .get_fmt = vidsw_get_format,
+ .set_fmt = vidsw_set_format,
+ .link_validate = vidsw_link_validate,
+};
+
+static struct v4l2_subdev_ops vidsw_subdev_ops = {
+ .pad = &vidsw_pad_ops,
+ .video = &vidsw_subdev_video_ops,
+};
+
+static int of_get_reg_field(struct device_node *node, struct reg_field *field)
+{
+ u32 bit_mask;
+ int ret;
+
+ ret = of_property_read_u32(node, "reg", &field->reg);
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct of_endpoint endpoint;
+ struct device_node *ep;
+ struct reg_field field;
+ struct vidsw *vidsw;
+ struct regmap *map;
+ unsigned int num_pads;
+ int ret;
+
+ vidsw = devm_kzalloc(&pdev->dev, sizeof(*vidsw), GFP_KERNEL);
+ if (!vidsw)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, vidsw);
+
+ v4l2_subdev_init(&vidsw->subdev, &vidsw_subdev_ops);
+ snprintf(vidsw->subdev.name, sizeof(vidsw->subdev.name), "%s",
+ np->name);
+ vidsw->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ vidsw->subdev.dev = &pdev->dev;
+
+ /* init default frame interval */
+ vidsw->timeperframe.numerator = 1;
+ vidsw->timeperframe.denominator = 30;
+
+ /*
+ * The largest numbered port is the output port. It determines
+ * total number of pads
+ */
+ num_pads = 0;
+ for_each_endpoint_of_node(np, ep) {
+ of_graph_parse_endpoint(ep, &endpoint);
+ num_pads = max(num_pads, endpoint.port + 1);
+ }
+
+ if (num_pads < 2) {
+ dev_err(&pdev->dev, "Not enough ports %d\n", num_pads);
+ if (!map) {
+ dev_err(&pdev->dev, "Failed to get syscon register map\n");
+ return PTR_ERR(map);
+ }
+
+ vidsw->field = devm_regmap_field_alloc(&pdev->dev, map, field);
+ if (IS_ERR(vidsw->field)) {
+ dev_err(&pdev->dev, "Failed to allocate regmap field\n");
+ return PTR_ERR(vidsw->field);
+ }
+
+ regmap_field_read(vidsw->field, &vidsw->active);
+ } else {
+ if (num_pads > 3) {
+ dev_err(&pdev->dev, "Too many ports %d\n", num_pads);
+ if (IS_ERR(vidsw->gpio)) {
+ dev_warn(&pdev->dev,
+ "could not request control gpio: %d\n", ret);
+ vidsw->gpio = NULL;
+ }
+
+ vidsw->active = gpiod_get_value(vidsw->gpio) ? 1 : 0;
+ }
+
+ vidsw->num_pads = num_pads;
+ vidsw->pads = devm_kzalloc(&pdev->dev, sizeof(*vidsw->pads) * num_pads,
+ GFP_KERNEL);
+ vidsw->format_mbus = devm_kzalloc(&pdev->dev,
+ sizeof(*vidsw->format_mbus) * num_pads, GFP_KERNEL);
+ vidsw->endpoint = devm_kzalloc(&pdev->dev,
+ sizeof(*vidsw->endpoint) * (num_pads - 1), GFP_KERNEL);
+
+ ret = vidsw_async_init(vidsw, np);
+{
+ struct vidsw *vidsw = platform_get_drvdata(pdev);
+ struct v4l2_subdev *sd = &vidsw->subdev;
+ { .compatible = "video-multiplexer", },
+
+static struct platform_driver vidsw_driver = {
+ .probe = vidsw_probe,
+ .remove = vidsw_remove,
+ .driver = {
+ .of_match_table = vidsw_dt_ids,
+ .name = "video-multiplexer",
+ },
+};
+
+module_platform_driver(vidsw_driver);
+
+MODULE_DESCRIPTION("video stream multiplexer");
+MODULE_AUTHOR("Sascha Hauer, Pengutronix");
+MODULE_AUTHOR("Philipp Zabel, Pengutronix");
This driver can handle SoC internal and external video bus multiplexers,
controlled either by register bit fields or by a GPIO. The subdevice
passes through frame interval and mbus configuration of the active input
to the output side.
Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
should be unregister.
- added media_entity_cleanup() to vidsw_remove().
- added missing MODULE_DEVICE_TABLE().
Suggested-by: Javier Martinez Canillas <jav...@dowhile0.org>
- there was a line left over from a previous iteration that negated
the new way of determining the pad count just before it which
has been removed (num_pads = of_get_child_count(np)).
- enforce that output frame interval must mirror input frame interval
in vidsw_s_frame_interval().
- init frame interval to a default 30 fps.
- add link_validate to media_entity_operations and v4l2_subdev_pad_ops.
- moved devicetree binding doc to a separate commit.
- Philipp Zabel has developed a set of patches that allow adding
to the subdev async notifier waiting list using a chaining method
from the async registered callbacks (v4l2_of_subdev_registered()
and the prep patches for that). For now, I've removed the use of
v4l2_of_subdev_registered() for the vidmux driver's registered
callback. This doesn't affect the functionality of this driver,
but allows for it to be merged now, before adding the chaining
support.
drivers/media/platform/Kconfig | 8 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/video-multiplexer.c | 498 +++++++++++++++++++++++++++++
drivers/media/platform/Makefile | 2 +
3 files changed, 508 insertions(+)
create mode 100644 drivers/media/platform/video-multiplexer.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 53f6f12..3226eb6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -74,6 +74,14 @@ config VIDEO_M32R_AR_M64278
To compile this driver as a module, choose M here: the
module will be called arv.
+config VIDEO_MULTIPLEXER
+ tristate "Video Multiplexer"
+ depends on VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
+ help
+ This driver provides support for SoC internal N:1 video bus
+ multiplexers controlled by register bitfields as well as external
+ 2:1 video multiplexers controlled by a single GPIO.
+
config VIDEO_OMAP3
tristate "OMAP 3 Camera support"
depends on VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API && ARCH_OMAP3
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 8959f6e..d418add 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -27,6 +27,8 @@ obj-$(CONFIG_VIDEO_SH_VEU) += sh_veu.o
obj-$(CONFIG_VIDEO_MEM2MEM_DEINTERLACE) += m2m-deinterlace.o
+obj-$(CONFIG_VIDEO_MULTIPLEXER) += video-multiplexer.o
+
obj-$(CONFIG_VIDEO_S3C_CAMIF) += s3c-camif/
obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS4_IS) += exynos4-is/
obj-$(CONFIG_VIDEO_SAMSUNG_S5P_JPEG) += s5p-jpeg/
diff --git a/drivers/media/platform/video-multiplexer.c b/drivers/media/platform/video-multiplexer.c
new file mode 100644
index 0000000..02a21072
--- /dev/null
+++ b/drivers/media/platform/video-multiplexer.c
@@ -0,0 +1,498 @@
+/*
+ * video stream multiplexer controlled via gpio or syscon
+ *
+ * Copyright (C) 2013 Pengutronix, Sascha Hauer <ker...@pengutronix.de>
+ * Copyright (C) 2016 Pengutronix, Philipp Zabel <ker...@pengutronix.de>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * This program is free software; you can redistribute it and/or
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/err.h>
+#include <linux/gpio/consumer.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-of.h>
+
+struct vidsw {
+ struct v4l2_subdev subdev;
+ unsigned int num_pads;
+ struct media_pad *pads;
+ struct v4l2_mbus_framefmt *format_mbus;
+ struct v4l2_fract timeperframe;
+ struct v4l2_of_endpoint *endpoint;
+ struct regmap_field *field;
+ struct gpio_desc *gpio;
+ int active;
+};
+
+static inline struct vidsw *v4l2_subdev_to_vidsw(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct vidsw, subdev);
+}
+
+static void vidsw_set_active(struct vidsw *vidsw, int active)
+{
+ vidsw->active = active;
+ if (active < 0)
+ return;
+
+ dev_dbg(vidsw->subdev.dev, "setting %d active\n", active);
+
+ if (vidsw->field)
+ regmap_field_write(vidsw->field, active);
+ else if (vidsw->gpio)
+ gpiod_set_value(vidsw->gpio, active);
+}
+
+static int vidsw_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ const struct media_pad *remote, u32 flags)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+
+ /* We have no limitations on enabling or disabling our output link */
+ if (local->index == vidsw->num_pads - 1)
+ return 0;
+
+ dev_dbg(sd->dev, "link setup %s -> %s", remote->entity->name,
+ local->entity->name);
+
+ if (!(flags & MEDIA_LNK_FL_ENABLED)) {
+
+ if (local->index == vidsw->active) {
+ dev_dbg(sd->dev, "going inactive\n");
+ vidsw->active = -1;
+ }
+ return 0;
+ }
+
+ if (vidsw->active >= 0) {
+ return 0;
+ }
+
+ struct media_pad *pad;
+
+ if (vidsw->active == local->index)
+ return 0;
+
+ pad = media_entity_remote_pad(&vidsw->pads[vidsw->active]);
+ if (pad) {
+ struct media_link *link;
+ int ret;
+
+ link = media_entity_find_link(pad,
+ &vidsw->pads[vidsw->active]);
+ if (link) {
+ ret = __media_entity_setup_link(link, 0);
+ if (ret)
+ return ret;
+ }
+ }
+ }
+
+ }
+ }
+ }
+ vidsw_set_active(vidsw, local->index);
+
+ return 0;
+}
+
+static struct media_entity_operations vidsw_ops = {
+ return 0;
+}
+
+ .link_setup = vidsw_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static bool vidsw_endpoint_disabled(struct device_node *ep)
+};
+
+{
+ struct device_node *rpp;
+
+ if (!of_device_is_available(ep))
+ return true;
+
+ rpp = of_graph_get_remote_port_parent(ep);
+ if (!rpp)
+ return true;
+
+ return !of_device_is_available(rpp);
+}
+
+static int vidsw_async_init(struct vidsw *vidsw, struct device_node *node)
+{
+ struct device_node *ep;
+ u32 portno;
+ int numports;
+ int ret;
+ int i;
+ bool active_link = false;
+
+ numports = vidsw->num_pads;
+
+ for (i = 0; i < numports - 1; i++)
+ vidsw->pads[i].flags = MEDIA_PAD_FL_SINK;
+ vidsw->pads[numports - 1].flags = MEDIA_PAD_FL_SOURCE;
+
+ vidsw->subdev.entity.function = MEDIA_ENT_F_VID_MUX;
+ ret = media_entity_pads_init(&vidsw->subdev.entity, numports,
+ vidsw->pads);
+ if (ret < 0)
+ return ret;
+
+ vidsw->subdev.entity.ops = &vidsw_ops;
+ return ret;
+
+
+ for_each_endpoint_of_node(node, ep) {
+ struct v4l2_of_endpoint endpoint;
+
+ v4l2_of_parse_endpoint(ep, &endpoint);
+
+ portno = endpoint.base.port;
+ if (portno >= numports - 1)
+ continue;
+
+ if (vidsw_endpoint_disabled(ep)) {
+ dev_dbg(vidsw->subdev.dev,
+ "port %d disabled\n", portno);
+ continue;
+ }
+
+ vidsw->endpoint[portno] = endpoint;
+
+ if (portno == vidsw->active)
+ active_link = true;
+ }
+
+ for (portno = 0; portno < numports - 1; portno++) {
+ if (!vidsw->endpoint[portno].base.local_node)
+ continue;
+
+ /* If the active input is not connected, use another */
+ if (!active_link) {
+ vidsw_set_active(vidsw, portno);
+ active_link = true;
+ }
+ }
+
+ return v4l2_async_register_subdev(&vidsw->subdev);
+}
+
+int vidsw_g_mbus_config(struct v4l2_subdev *sd, struct v4l2_mbus_config *cfg)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+ struct media_pad *pad;
+ int ret;
+
+ if (vidsw->active == -1) {
+ dev_err(sd->dev, "no configuration for inactive mux\n");
+ return -EINVAL;
+ }
+
+ /*
+ }
+
+ * Retrieve media bus configuration from the entity connected to the
+ * active input
+ */
+ pad = media_entity_remote_pad(&vidsw->pads[vidsw->active]);
+ if (pad) {
+ sd = media_entity_to_v4l2_subdev(pad->entity);
+ ret = v4l2_subdev_call(sd, video, g_mbus_config, cfg);
+ if (ret == -ENOIOCTLCMD)
+ pad = NULL;
+ else if (ret < 0) {
+ dev_err(sd->dev, "failed to get source configuration\n");
+ return ret;
+ }
+ }
+ if (!pad) {
+ }
+ }
+ /* Mirror the input side on the output side */
+ cfg->type = vidsw->endpoint[vidsw->active].bus_type;
+ if (cfg->type == V4L2_MBUS_PARALLEL ||
+ cfg->type == V4L2_MBUS_BT656)
+ cfg->flags = vidsw->endpoint[vidsw->active].bus.parallel.flags;
+ }
+
+ return 0;
+}
+
+static int vidsw_s_stream(struct v4l2_subdev *sd, int enable)
+
+ return 0;
+}
+
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+ struct v4l2_subdev *upstream_sd;
+ struct media_pad *pad;
+
+ if (vidsw->active == -1) {
+ dev_err(sd->dev, "Can not start streaming on inactive mux\n");
+ return -EINVAL;
+ }
+
+ pad = media_entity_remote_pad(&sd->entity.pads[vidsw->active]);
+ }
+
+ if (!pad) {
+ dev_err(sd->dev, "Failed to find remote source pad\n");
+ return -ENOLINK;
+ }
+
+ if (!is_media_entity_v4l2_subdev(pad->entity)) {
+ dev_err(sd->dev, "Upstream entity is not a v4l2 subdev\n");
+ return -ENODEV;
+ }
+
+ upstream_sd = media_entity_to_v4l2_subdev(pad->entity);
+ }
+
+
+ return v4l2_subdev_call(upstream_sd, video, s_stream, enable);
+}
+
+static int vidsw_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+{
+
+ fi->interval = vidsw->timeperframe;
+
+ return 0;
+}
+
+static int vidsw_s_frame_interval(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+{
+
+ /* Output pad mirrors active input pad, no limits on input pads */
+ if (fi->pad == (vidsw->num_pads - 1))
+ fi->interval = vidsw->timeperframe;
+
+ vidsw->timeperframe = fi->interval;
+
+ return 0;
+}
+
+static const struct v4l2_subdev_video_ops vidsw_subdev_video_ops = {
+ return 0;
+}
+
+ .g_mbus_config = vidsw_g_mbus_config,
+ .s_stream = vidsw_s_stream,
+ .g_frame_interval = vidsw_g_frame_interval,
+ .s_frame_interval = vidsw_s_frame_interval,
+};
+
+static struct v4l2_mbus_framefmt *
+__vidsw_get_pad_format(struct v4l2_subdev *sd,
+
+static struct v4l2_mbus_framefmt *
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad, u32 which)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+
+ switch (which) {
+ case V4L2_SUBDEV_FORMAT_TRY:
+ return v4l2_subdev_get_try_format(sd, cfg, pad);
+ case V4L2_SUBDEV_FORMAT_ACTIVE:
+ return &vidsw->format_mbus[pad];
+ default:
+ return NULL;
+ }
+}
+
+static int vidsw_get_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ sdformat->format = *__vidsw_get_pad_format(sd, cfg, sdformat->pad,
+ struct v4l2_subdev_format *sdformat)
+{
+ sdformat->which);
+ return 0;
+}
+
+static int vidsw_set_format(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct vidsw *vidsw = v4l2_subdev_to_vidsw(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *mbusformat;
+
+ if (sdformat->pad >= vidsw->num_pads)
+ return -EINVAL;
+
+ mbusformat = __vidsw_get_pad_format(sd, cfg, sdformat->pad,
+ sdformat->which);
+ if (!mbusformat)
+ return -EINVAL;
+
+ /* Output pad mirrors active input pad, no limitations on input pads */
+ if (sdformat->pad == (vidsw->num_pads - 1) && vidsw->active >= 0)
+ sdformat->format = vidsw->format_mbus[vidsw->active];
+
+ *mbusformat = sdformat->format;
+
+ return 0;
+}
+
+static int vidsw_link_validate(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ int ret;
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ return v4l2_subdev_link_validate_frame_interval(link);
+ return ret;
+
+}
+
+static struct v4l2_subdev_pad_ops vidsw_pad_ops = {
+ .get_fmt = vidsw_get_format,
+ .set_fmt = vidsw_set_format,
+ .link_validate = vidsw_link_validate,
+};
+
+static struct v4l2_subdev_ops vidsw_subdev_ops = {
+ .pad = &vidsw_pad_ops,
+ .video = &vidsw_subdev_video_ops,
+};
+
+static int of_get_reg_field(struct device_node *node, struct reg_field *field)
+{
+ u32 bit_mask;
+ int ret;
+
+ ret = of_property_read_u32(node, "reg", &field->reg);
+ if (ret < 0)
+ return ret;
+
+ ret = of_property_read_u32(node, "bit-mask", &bit_mask);
+ return ret;
+
+ if (ret < 0)
+ return ret;
+
+ ret = of_property_read_u32(node, "bit-shift", &field->lsb);
+ return ret;
+
+ if (ret < 0)
+ return ret;
+
+ field->msb = field->lsb + fls(bit_mask) - 1;
+ return ret;
+
+
+ return 0;
+}
+
+static int vidsw_probe(struct platform_device *pdev)
+ return 0;
+}
+
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct of_endpoint endpoint;
+ struct device_node *ep;
+ struct reg_field field;
+ struct vidsw *vidsw;
+ struct regmap *map;
+ unsigned int num_pads;
+ int ret;
+
+ vidsw = devm_kzalloc(&pdev->dev, sizeof(*vidsw), GFP_KERNEL);
+ if (!vidsw)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, vidsw);
+
+ v4l2_subdev_init(&vidsw->subdev, &vidsw_subdev_ops);
+ snprintf(vidsw->subdev.name, sizeof(vidsw->subdev.name), "%s",
+ np->name);
+ vidsw->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ vidsw->subdev.dev = &pdev->dev;
+
+ /* init default frame interval */
+ vidsw->timeperframe.numerator = 1;
+ vidsw->timeperframe.denominator = 30;
+
+ /*
+ * The largest numbered port is the output port. It determines
+ * total number of pads
+ */
+ num_pads = 0;
+ for_each_endpoint_of_node(np, ep) {
+ of_graph_parse_endpoint(ep, &endpoint);
+ num_pads = max(num_pads, endpoint.port + 1);
+ }
+
+ if (num_pads < 2) {
+ dev_err(&pdev->dev, "Not enough ports %d\n", num_pads);
+ return -EINVAL;
+ }
+
+ ret = of_get_reg_field(np, &field);
+ }
+
+ if (ret == 0) {
+ map = syscon_node_to_regmap(np->parent);
+ if (!map) {
+ dev_err(&pdev->dev, "Failed to get syscon register map\n");
+ return PTR_ERR(map);
+ }
+
+ vidsw->field = devm_regmap_field_alloc(&pdev->dev, map, field);
+ if (IS_ERR(vidsw->field)) {
+ dev_err(&pdev->dev, "Failed to allocate regmap field\n");
+ return PTR_ERR(vidsw->field);
+ }
+
+ regmap_field_read(vidsw->field, &vidsw->active);
+ } else {
+ if (num_pads > 3) {
+ dev_err(&pdev->dev, "Too many ports %d\n", num_pads);
+ return -EINVAL;
+ }
+
+ vidsw->gpio = devm_gpiod_get(&pdev->dev, NULL, GPIOD_OUT_LOW);
+ }
+
+ if (IS_ERR(vidsw->gpio)) {
+ dev_warn(&pdev->dev,
+ "could not request control gpio: %d\n", ret);
+ vidsw->gpio = NULL;
+ }
+
+ vidsw->active = gpiod_get_value(vidsw->gpio) ? 1 : 0;
+ }
+
+ vidsw->num_pads = num_pads;
+ vidsw->pads = devm_kzalloc(&pdev->dev, sizeof(*vidsw->pads) * num_pads,
+ GFP_KERNEL);
+ vidsw->format_mbus = devm_kzalloc(&pdev->dev,
+ sizeof(*vidsw->format_mbus) * num_pads, GFP_KERNEL);
+ vidsw->endpoint = devm_kzalloc(&pdev->dev,
+ sizeof(*vidsw->endpoint) * (num_pads - 1), GFP_KERNEL);
+
+ ret = vidsw_async_init(vidsw, np);
+ if (ret)
+ return ret;
+
+ return ret;
+
+ return 0;
+}
+
+static int vidsw_remove(struct platform_device *pdev)
+}
+
+{
+ struct vidsw *vidsw = platform_get_drvdata(pdev);
+ struct v4l2_subdev *sd = &vidsw->subdev;
+
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct of_device_id vidsw_dt_ids[] = {
+ return 0;
+}
+
+ { .compatible = "video-multiplexer", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, vidsw_dt_ids);
+};
+
+static struct platform_driver vidsw_driver = {
+ .probe = vidsw_probe,
+ .remove = vidsw_remove,
+ .driver = {
+ .of_match_table = vidsw_dt_ids,
+ .name = "video-multiplexer",
+ },
+};
+
+module_platform_driver(vidsw_driver);
+
+MODULE_DESCRIPTION("video stream multiplexer");
+MODULE_AUTHOR("Sascha Hauer, Pengutronix");
+MODULE_AUTHOR("Philipp Zabel, Pengutronix");
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
From: Russell King <rmk+k...@armlinux.org.uk>
Bayer formats must be treated as generic data and passthrough mode must
be used. Add the correct setup for these formats.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
- added check to csi_link_validate() to verify that destination is
IDMAC output pad when passthrough conditions exist: bayer formats
and 16-bit parallel buses.
drivers/staging/media/imx/imx-media-csi.c | 52 +++++++++++++++++++++++++------
1 file changed, 43 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index ab78ff7..a7d04e4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -276,10 +276,11 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
struct v4l2_mbus_framefmt *infmt;
- unsigned int burst_size;
struct ipu_image image;
+ u32 passthrough_bits;
dma_addr_t phys[2];
bool passthrough;
+ u32 burst_size;
int ret;
infmt = &priv->format_mbus[CSI_SINK_PAD];
@@ -306,15 +307,38 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
/*
- * If the sensor uses 16-bit parallel CSI bus, we must handle
- * the data internally in the IPU as 16-bit generic, aka
- * passthrough mode.
+ * Check for conditions that require the IPU to handle the
+ * data internally as generic data, aka passthrough mode:
+ * - raw bayer formats
+ * - the sensor bus is 16-bit parallel
*/
- passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
- sensor_ep->bus.parallel.bus_width >= 16);
+ switch (image.pix.pixelformat) {
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ burst_size = 8;
+ passthrough = true;
+ passthrough_bits = 8;
+ break;
+ case V4L2_PIX_FMT_SBGGR16:
+ case V4L2_PIX_FMT_SGBRG16:
+ case V4L2_PIX_FMT_SGRBG16:
+ case V4L2_PIX_FMT_SRGGB16:
+ burst_size = 4;
+ passthrough = true;
+ passthrough_bits = 16;
+ break;
+ default:
+ passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+ sensor_ep->bus.parallel.bus_width >= 16);
+ passthrough_bits = 16;
+ break;
+ }
if (passthrough)
- ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+ ipu_cpmem_set_format_passthrough(priv->idmac_ch,
+ passthrough_bits);
/*
* Set the channel for the direct CSI-->memory via SMFC
@@ -725,6 +749,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt)
{
struct imx_media_subdev *sensor;
bool is_csi2;
@@ -749,8 +774,17 @@ static int csi_link_validate(struct v4l2_subdev *sd,
priv->sensor = sensor;
sensor_ep = &priv->sensor->sensor_ep;
-
is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+ incc = priv->cc[CSI_SINK_PAD];
+
+ if (priv->dest != IPU_CSI_DEST_IDMAC &&
+ (incc->bayer || (!is_csi2 &&
+ sensor_ep->bus.parallel.bus_width >= 16))) {
+ v4l2_err(&priv->sd,
+ "bayer/16-bit parallel buses must go to IDMAC pad\n");
+ ret = -EINVAL;
+ goto out;
+ }
if (is_csi2) {
int vc_num = 0;
@@ -775,7 +809,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
/* select either parallel or MIPI-CSI2 as input to CSI */
ipu_set_csi_src_mux(priv->ipu, priv->csi_id, is_csi2);
-
+out:
Bayer formats must be treated as generic data and passthrough mode must
be used. Add the correct setup for these formats.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
- added check to csi_link_validate() to verify that destination is
IDMAC output pad when passthrough conditions exist: bayer formats
and 16-bit parallel buses.
drivers/staging/media/imx/imx-media-csi.c | 52 +++++++++++++++++++++++++------
1 file changed, 43 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index ab78ff7..a7d04e4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -276,10 +276,11 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
struct imx_media_video_dev *vdev = priv->vdev;
struct v4l2_of_endpoint *sensor_ep;
struct v4l2_mbus_framefmt *infmt;
- unsigned int burst_size;
struct ipu_image image;
+ u32 passthrough_bits;
dma_addr_t phys[2];
bool passthrough;
+ u32 burst_size;
int ret;
infmt = &priv->format_mbus[CSI_SINK_PAD];
@@ -306,15 +307,38 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
/*
- * If the sensor uses 16-bit parallel CSI bus, we must handle
- * the data internally in the IPU as 16-bit generic, aka
- * passthrough mode.
+ * Check for conditions that require the IPU to handle the
+ * data internally as generic data, aka passthrough mode:
+ * - raw bayer formats
+ * - the sensor bus is 16-bit parallel
*/
- passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
- sensor_ep->bus.parallel.bus_width >= 16);
+ switch (image.pix.pixelformat) {
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ burst_size = 8;
+ passthrough = true;
+ passthrough_bits = 8;
+ break;
+ case V4L2_PIX_FMT_SBGGR16:
+ case V4L2_PIX_FMT_SGBRG16:
+ case V4L2_PIX_FMT_SGRBG16:
+ case V4L2_PIX_FMT_SRGGB16:
+ burst_size = 4;
+ passthrough = true;
+ passthrough_bits = 16;
+ break;
+ default:
+ passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+ sensor_ep->bus.parallel.bus_width >= 16);
+ passthrough_bits = 16;
+ break;
+ }
if (passthrough)
- ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+ ipu_cpmem_set_format_passthrough(priv->idmac_ch,
+ passthrough_bits);
/*
* Set the channel for the direct CSI-->memory via SMFC
@@ -725,6 +749,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt)
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ const struct imx_media_pixfmt *incc;
struct v4l2_of_endpoint *sensor_ep;
struct imx_media_subdev *sensor;
bool is_csi2;
@@ -749,8 +774,17 @@ static int csi_link_validate(struct v4l2_subdev *sd,
priv->sensor = sensor;
sensor_ep = &priv->sensor->sensor_ep;
-
is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+ incc = priv->cc[CSI_SINK_PAD];
+
+ if (priv->dest != IPU_CSI_DEST_IDMAC &&
+ (incc->bayer || (!is_csi2 &&
+ sensor_ep->bus.parallel.bus_width >= 16))) {
+ v4l2_err(&priv->sd,
+ "bayer/16-bit parallel buses must go to IDMAC pad\n");
+ ret = -EINVAL;
+ goto out;
+ }
if (is_csi2) {
int vc_num = 0;
@@ -775,7 +809,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
/* select either parallel or MIPI-CSI2 as input to CSI */
ipu_set_csi_src_mux(priv->ipu, priv->csi_id, is_csi2);
-
+out:
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
This is a media entity subdevice for the i.MX Camera
Sensor Interface module.
drivers/staging/media/imx/Kconfig | 14 +
drivers/staging/media/imx/Makefile | 2 +
drivers/staging/media/imx/imx-media-csi.c | 1311 +++++++++++++++++++++++++++++
3 files changed, 1327 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-csi.c
diff --git a/drivers/staging/media/imx/Kconfig b/drivers/staging/media/imx/Kconfig
index 62a3c34..e27ad6d 100644
--- a/drivers/staging/media/imx/Kconfig
+++ b/drivers/staging/media/imx/Kconfig
@@ -4,3 +4,17 @@ config VIDEO_IMX_MEDIA
---help---
Say yes here to enable support for video4linux media controller
driver for the i.MX5/6 SOC.
+
+if VIDEO_IMX_MEDIA
+menu "i.MX5/6 Media Sub devices"
+
+config VIDEO_IMX_CSI
+ tristate "i.MX5/6 Camera Sensor Interface driver"
+ depends on VIDEO_IMX_MEDIA && VIDEO_DEV && I2C
+ select VIDEOBUF2_DMA_CONTIG
+ default y
+ ---help---
+ A video4linux camera sensor interface driver for i.MX5/6.
+
+endmenu
+endif
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 4606a3a..c054490 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,3 +4,5 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
+
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
new file mode 100644
index 0000000..ab78ff7
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -0,0 +1,1311 @@
+/*
+ * V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
+ *
+ */
+#include <media/v4l2-of.h>
+ * We allow planar output, so we have to align width by 16 pixels
+ * to meet IDMAC alignment requirements.
+ * alignment.
+ struct device *dev;
+ struct ipu_soc *ipu;
+ struct imx_media_dev *md;
+ struct v4l2_subdev sd;
+ struct media_pad pad[CSI_NUM_PADS];
+ /* the video device at IDMAC output pad */
+ struct imx_media_video_dev *vdev;
+ struct imx_media_fim *fim;
+ int csi_id;
+ int smfc_id;
+
+ struct ipuv3_channel *idmac_ch;
+ struct ipu_smfc *smfc;
+ struct ipu_csi *csi;
+
+ struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
+ const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
+
+ int vc_num;
+
+ /* the attached sensor at stream on */
+
+static inline struct csi_priv *sd_to_dev(struct v4l2_subdev *sdev)
+{
+ return container_of(sdev, struct csi_priv, sd);
+}
+
+static void csi_idmac_put_ipu_resources(struct csi_priv *priv)
+{
+ if (!IS_ERR_OR_NULL(priv->idmac_ch))
+ ipu_idmac_put(priv->idmac_ch);
+ priv->idmac_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->smfc))
+ ipu_smfc_put(priv->smfc);
+ priv->smfc = NULL;
+}
+
+static int csi_idmac_get_ipu_resources(struct csi_priv *priv)
+{
+ int ch_num, ret;
+
+ ch_num = IPUV3_CHANNEL_CSI0 + priv->smfc_id;
+
+ priv->smfc = ipu_smfc_get(priv->ipu, ch_num);
+ if (IS_ERR(priv->smfc)) {
+ v4l2_err(&priv->sd, "failed to get SMFC\n");
+ ret = PTR_ERR(priv->smfc);
+ if (IS_ERR(priv->idmac_ch)) {
+ ret = PTR_ERR(priv->idmac_ch);
+ goto out;
+ csi_idmac_put_ipu_resources(priv);
+ }
+
+ if (ipu_idmac_buffer_is_ready(priv->idmac_ch, priv->ipu_buf_num))
+ ipu_idmac_clear_buffer(priv->idmac_ch, priv->ipu_buf_num);
+
+ ipu_cpmem_set_buffer(priv->idmac_ch, priv->ipu_buf_num, phys);
+}
+
+static void csi_call_fim(struct csi_priv *priv)
+{
+ if (priv->fim) {
+ struct timespec cur_ts;
+
+ ktime_get_ts(&cur_ts);
+ /* call frame interval monitor */
+ imx_media_fim_eof_monitor(priv->fim, &cur_ts);
+ }
+}
+
+static irqreturn_t csi_idmac_eof_interrupt(int irq, void *dev_id)
+{
+ struct csi_priv *priv = dev_id;
+
+ csi_vb2_buf_done(priv);
+{
+ struct csi_priv *priv = dev_id;
+
+ v4l2_subdev_notify_event(&priv->sd, &ev);
+
+ return IRQ_HANDLED;
+}
+
+/*
+{
+ struct csi_priv *priv = (struct csi_priv *)data;
+ struct imx_media_video_dev *vdev = priv->vdev;
+
+ v4l2_err(&priv->sd, "EOF timeout\n");
+}
+
+static void csi_idmac_unsetup_vb2_buf(struct csi_priv *priv,
+ enum vb2_buffer_state return_state)
+{
+ struct imx_media_buffer *buf;
+ int i;
+
+static int csi_idmac_setup_channel(struct csi_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_of_endpoint *sensor_ep;
+ struct v4l2_mbus_framefmt *infmt;
+ bool passthrough;
+ int ret;
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ ipu_cpmem_zero(priv->idmac_ch);
+
+ image.phys0 = phys[0];
+ image.phys1 = phys[1];
+
+ ret = ipu_cpmem_set_image(priv->idmac_ch, &image);
+
+ /*
+ * If the sensor uses 16-bit parallel CSI bus, we must handle
+ * the data internally in the IPU as 16-bit generic, aka
+ * passthrough mode.
+ */
+ ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+
+ /*
+ * Set the channel for the direct CSI-->memory via SMFC
+ * use-case to very high priority, by enabling the watermark
+ * signal in the SMFC, enabling WM in the channel, and setting
+ * the channel priority to high.
+ *
+ * Refer to the i.mx6 rev. D TRM Table 36-8: Calculated priority
+ * value.
+ *
+ * The WM's are set very low by intention here to ensure that
+ * the SMFC FIFOs do not overflow.
+ */
+ ipu_smfc_set_watermark(priv->smfc, 0x02, 0x01);
+ ipu_cpmem_set_high_priority(priv->idmac_ch);
+ ipu_idmac_enable_watermark(priv->idmac_ch, true);
+ ipu_cpmem_set_axi_id(priv->idmac_ch, 0);
+ ipu_idmac_lock_enable(priv->idmac_ch, 8);
+
+ burst_size = ipu_cpmem_get_burstsize(priv->idmac_ch);
+ burst_size = passthrough ?
+ (burst_size >> 3) - 1 : (burst_size >> 2) - 1;
+
+ ipu_smfc_set_burstsize(priv->smfc, burst_size);
+ ipu_cpmem_interlaced_scan(priv->idmac_ch,
+ image.pix.bytesperline);
+
+ ipu_idmac_set_double_buffer(priv->idmac_ch, true);
+ ipu_smfc_disable(priv->smfc);
+
+ csi_idmac_unsetup_vb2_buf(priv, state);
+}
+
+static int csi_idmac_setup(struct csi_priv *priv)
+ ipu_dump(priv->ipu);
+
+ ipu_smfc_enable(priv->smfc);
+ ipu_idmac_select_buffer(priv->idmac_ch, 1);
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_pix_format *outfmt;
+ int ret;
+
+ ret = csi_idmac_get_ipu_resources(priv);
+ if (ret) {
+ v4l2_err(&priv->sd, "csi_idmac_setup failed: %d\n", ret);
+ goto out_free_dma_buf;
+ }
+ IPU_IRQ_NFB4EOF);
+ ret = devm_request_irq(priv->dev, priv->nfb4eof_irq,
+ csi_idmac_nfb4eof_interrupt, 0,
+ "imx-smfc-nfb4eof", priv);
+ if (ret) {
+ v4l2_err(&priv->sd,
+ IPU_IRQ_EOF);
+
+ ret = devm_request_irq(priv->dev, priv->eof_irq,
+ csi_idmac_eof_interrupt, 0,
+ "imx-smfc-eof", priv);
+ if (ret) {
+ v4l2_err(&priv->sd,
+ devm_free_irq(priv->dev, priv->nfb4eof_irq, priv);
+out_unsetup:
+ csi_idmac_unsetup(priv, VB2_BUF_STATE_QUEUED);
+out_free_dma_buf:
+{
+ unsigned long flags;
+ int ret;
+
+ /*
+
+ devm_free_irq(priv->dev, priv->eof_irq, priv);
+ devm_free_irq(priv->dev, priv->nfb4eof_irq, priv);
+
+ csi_idmac_unsetup(priv, VB2_BUF_STATE_ERROR);
+}
+
+/* Update the CSI whole sensor and active windows */
+static int csi_setup(struct csi_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt, *outfmt;
+ struct v4l2_mbus_config sensor_mbus_cfg;
+ struct v4l2_of_endpoint *sensor_ep;
+ struct v4l2_mbus_framefmt if_fmt;
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+ outfmt = &priv->format_mbus[priv->active_output_pad];
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ /* compose mbus_config from sensor endpoint */
+ sensor_mbus_cfg.type = sensor_ep->bus_type;
+ sensor_mbus_cfg.flags = (sensor_ep->bus_type == V4L2_MBUS_CSI2) ?
+ sensor_ep->bus.mipi_csi2.flags :
+ sensor_ep->bus.parallel.flags;
+
+ /*
+ * we need to pass input sensor frame to CSI interface, but
+ * with translated field type from output format
+ */
+ if_fmt = *infmt;
+ if_fmt.field = outfmt->field;
+
+ ipu_csi_set_window(priv->csi, &priv->crop);
+
+ ipu_csi_set_dest(priv->csi, priv->dest);
+
+ ipu_csi_dump(priv->csi);
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return -EINVAL;
+ }
+
+ if (priv->dest == IPU_CSI_DEST_IDMAC) {
+ ret = csi_idmac_start(priv);
+ if (ret)
+ goto idmac_stop;
+
+ /* start the frame interval monitor */
+ if (priv->fim) {
+ ret = imx_media_fim_set_stream(priv->fim, priv->sensor, true);
+ if (ret)
+ goto idmac_stop;
+ }
+
+ ret = ipu_csi_enable(priv->csi);
+ if (ret) {
+ v4l2_err(&priv->sd, "CSI enable error: %d\n", ret);
+ goto fim_off;
+ if (priv->fim)
+ imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+idmac_stop:
+ if (priv->dest == IPU_CSI_DEST_IDMAC)
+ csi_idmac_stop(priv);
+{
+ if (priv->dest == IPU_CSI_DEST_IDMAC)
+ csi_idmac_stop(priv);
+
+ /* stop the frame interval monitor */
+ if (priv->fim)
+ imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+
+ ipu_csi_disable(priv->csi);
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int csi_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+ mutex_lock(&priv->lock);
+
+ /*
+ if (priv->fim) {
+ ret = imx_media_fim_set_power(priv->fim, on);
+ if (ret)
+ WARN_ON(priv->power_count < 0);
+out:
+ mutex_unlock(&priv->lock);
+ struct v4l2_subdev *remote_sd;
+
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ priv->active_output_pad = local->index;
+
+ /* set CSI destination */
+ if (local->index == CSI_SRC_PAD_IDMAC) {
+ if (!is_media_entity_v4l2_video_device(remote->entity)) {
+ priv->dest = IPU_CSI_DEST_IDMAC;
+ case IMX_MEDIA_GRP_ID_VDIC:
+ priv->dest = IPU_CSI_DEST_VDIC;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRP:
+ priv->dest = IPU_CSI_DEST_IC;
+ break;
+ default:
+ struct v4l2_of_endpoint *sensor_ep;
+ struct imx_media_subdev *sensor;
+ bool is_csi2;
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+
+ if (is_csi2) {
+ int vc_num = 0;
+ /*
+ * NOTE! It seems the virtual channels from the mipi csi-2
+ * receiver are used only for routing by the video mux's,
+ * or for hard-wired routing to the CSI's. Once the stream
+ * enters the CSI's however, they are treated internally
+ * in the IPU as virtual channel 0.
+ */
+#if 0
+ mutex_unlock(&priv->lock);
+ vc_num = imx_media_find_mipi_csi2_channel(priv->md,
+ &priv->sd.entity);
+ if (vc_num < 0)
+ return vc_num;
+ mutex_lock(&priv->lock);
+#endif
+ ipu_csi_set_mipi_datatype(priv->csi, vc_num,
+ &priv->format_mbus[CSI_SINK_PAD]);
+ }
+
+ /* select either parallel or MIPI-CSI2 as input to CSI */
+ ipu_set_csi_src_mux(priv->ipu, priv->csi_id, is_csi2);
+
+ mutex_unlock(&priv->lock);
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ csi_call_fim(priv);
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+ struct imx_media_subdev *sensor)
+{
+ struct v4l2_of_endpoint *sensor_ep;
+ sensor_ep = &sensor->sensor_ep;
+
+ crop->width = min_t(__u32, infmt->width, crop->width);
+ if (crop->left + crop->width > infmt->width)
+ crop->left = infmt->width - crop->width;
+ /* adjust crop left/width to h/w alignment restrictions */
+ crop->left &= ~0x3;
+ crop->width &= ~0x7;
+
+ /*
+ * FIXME: not sure why yet, but on interlaced bt.656,
+ * changing the vertical cropping causes loss of vertical
+ * sync, so fix it to NTSC/PAL active lines. NTSC contains
+ * 2 extra lines of active video that need to be cropped.
+ */
+ if (sensor_ep->bus_type == V4L2_MBUS_BT656 &&
+ (V4L2_FIELD_HAS_BOTH(infmt->field) ||
+ infmt->field == V4L2_FIELD_ALTERNATE)) {
+ crop->height = infmt->height;
+ crop->top = (infmt->height == 480) ? 2 : 0;
+ } else {
+ crop->height = min_t(__u32, infmt->height, crop->height);
+ if (crop->top + crop->height > infmt->height)
+ crop->top = infmt->height - crop->height;
+ struct v4l2_subdev_pad_config *cfg,
+ return -EINVAL;
+
+ if (code->pad == CSI_SRC_PAD_DIRECT)
+
+
+ if (sdformat->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ sdformat->format = priv->format_mbus[sdformat->pad];
+
+ mutex_unlock(&priv->lock);
+ const struct imx_media_pixfmt *cc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct imx_media_subdev *sensor;
+ struct v4l2_rect crop;
+
+ if (sdformat->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
+
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+ case CSI_SRC_PAD_IDMAC:
+ crop.left = priv->crop.left;
+ crop.top = priv->crop.top;
+ goto out;
+ sdformat->format.width = crop.width;
+ sdformat->format.height = crop.height;
+
+ if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
+ sdformat->format.code = cc->codes[0];
+ }
+
+ sdformat->format.code = infmt->code;
+ cc = imx_media_find_format(
+ 0, sdformat->format.code,
+ cc = imx_media_find_ipu_format(0, sdformat->format.code,
+ true);
+ if (!cc) {
+ }
+
+
+ /*
+ * translate V4L2_FIELD_ALTERNATE to SEQ_TB or SEQ_BT
+ * depending on input height (assume NTSC top-bottom
+ * order if 480 lines, otherwise PAL bottom-top order).
+ */
+ if (sdformat->format.field == V4L2_FIELD_ALTERNATE) {
+ sdformat->format.field = (infmt->height == 480) ?
+ V4L2_FIELD_SEQ_TB : V4L2_FIELD_SEQ_BT;
+ }
+ break;
+ case CSI_SINK_PAD:
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0, true, false);
+ cc = imx_media_find_format(0, code, true, false);
+ sdformat->format.code = cc->codes[0];
+ }
+ break;
+ default:
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ if (sdformat->pad == CSI_SRC_PAD_DIRECT ||
+ sdformat->pad == CSI_SRC_PAD_IDMAC)
+ mutex_unlock(&priv->lock);
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *infmt;
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ sel->r.left = 0;
+ sel->r.top = 0;
+ sel->r.width = infmt->width;
+ sel->r.height = infmt->height;
+ break;
+ case V4L2_SEL_TGT_CROP:
+ sel->r = priv->crop;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ mutex_unlock(&priv->lock);
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *outfmt;
+ struct imx_media_subdev *sensor;
+ sel->pad == CSI_SINK_PAD ||
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ }
+
+ /*
+ * Modifying the crop rectangle always changes the format on the source
+ * pad. If the KEEP_CONFIG flag is set, just return the current crop
+ * rectangle.
+ */
+ if (sel->flags & V4L2_SEL_FLAG_KEEP_CONFIG) {
+ sel->r = priv->crop;
+ if (sel->which == V4L2_SUBDEV_FORMAT_TRY)
+ cfg->try_crop = sel->r;
+
+ ret = csi_try_crop(priv, &sel->r, sensor);
+ if (ret)
+ goto out;
+
+ if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_crop = sel->r;
+ } else {
+ priv->crop = sel->r;
+ /* Update the source format */
+ outfmt->width = sel->r.width;
+ outfmt->height = sel->r.height;
+ sub->type != V4L2_EVENT_FRAME_INTERVAL_ERROR)
+
+
+/*
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ priv->csi = ipu_csi_get(priv->ipu, priv->csi_id);
+ if (IS_ERR(priv->csi)) {
+ v4l2_err(&priv->sd, "failed to get CSI%d\n", priv->csi_id);
+ return PTR_ERR(priv->csi);
+ }
+
+ for (i = 0; i < CSI_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == CSI_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+ }
+
+ priv->fim = imx_media_fim_init(&priv->sd);
+ if (IS_ERR(priv->fim)) {
+ ret = PTR_ERR(priv->fim);
+ goto put_csi;
+ }
+
+ ret = media_entity_pads_init(&sd->entity, CSI_NUM_PADS, priv->pad);
+ if (ret)
+ goto free_fim;
+ goto free_fim;
+ if (priv->fim)
+ imx_media_fim_free(priv->fim);
+put_csi:
+ ipu_csi_put(priv->csi);
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+ imx_media_capture_device_unregister(priv->vdev);
+
+ if (priv->fim)
+ imx_media_fim_free(priv->fim);
+
+ if (!IS_ERR_OR_NULL(priv->csi))
+ ipu_csi_put(priv->csi);
+}
+
+static struct media_entity_operations csi_entity_ops = {
+ .link_setup = csi_link_setup,
+ .s_power = csi_s_power,
+ .interrupt_service_routine = csi_eof_isr,
+ .subscribe_event = csi_subscribe_event,
+ .unsubscribe_event = csi_unsubscribe_event,
+};
+
+static struct v4l2_subdev_video_ops csi_video_ops = {
+ .s_stream = csi_s_stream,
+};
+
+static struct v4l2_subdev_pad_ops csi_pad_ops = {
+ .enum_mbus_code = csi_enum_mbus_code,
+ .get_fmt = csi_get_fmt,
+ .set_fmt = csi_set_fmt,
+ .get_selection = csi_get_selection,
+ .set_selection = csi_set_selection,
+ .link_validate = csi_link_validate,
+};
+
+static struct v4l2_subdev_ops csi_subdev_ops = {
+ .core = &csi_core_ops,
+ .video = &csi_video_ops,
+ .pad = &csi_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops csi_internal_ops = {
+ .registered = csi_registered,
+ .unregistered = csi_unregistered,
+};
+
+static int imx_csi_probe(struct platform_device *pdev)
+{
+ struct ipu_client_platformdata *pdata;
+ struct pinctrl *pinctrl;
+ struct csi_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ ret = dma_set_coherent_mask(priv->dev, DMA_BIT_MASK(32));
+ priv->ipu = dev_get_drvdata(priv->dev->parent);
+
+ /* get our CSI id */
+ priv->smfc_id = (priv->csi_id == 0) ? 0 : 2;
+
+ spin_lock_init(&priv->irqlock);
+
+ v4l2_subdev_init(&priv->sd, &csi_subdev_ops);
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = &csi_internal_ops;
+ priv->sd.entity.ops = &csi_entity_ops;
+ priv->sd.of_node = pdata->of_node;
+ IMX_MEDIA_GRP_ID_CSI1 : IMX_MEDIA_GRP_ID_CSI0;
+ imx_media_grp_id_to_sd_name(priv->sd.name, sizeof(priv->sd.name),
+ priv->sd.grp_id, ipu_get_num(priv->ipu));
+
+ priv->vdev = imx_media_capture_device_init(&priv->sd,
+ CSI_SRC_PAD_IDMAC);
+
+ /*
+ * The IPUv3 driver did not assign an of_node to this
+ * device. As a result, pinctrl does not automatically
+ * configure our pin groups, so we need to do that manually
+ * here, after setting this device's of_node.
+ */
+ priv->dev->of_node = pdata->of_node;
+ pinctrl = devm_pinctrl_get_select_default(priv->dev);
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct csi_priv *priv = sd_to_dev(sd);
+
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+ { .name = "imx-ipuv3-csi" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_csi_ids);
+
+static struct platform_driver imx_csi_driver = {
+ .probe = imx_csi_probe,
+ .remove = imx_csi_remove,
+ .id_table = imx_csi_ids,
+ .driver = {
+ .name = "imx-ipuv3-csi",
+ },
+};
+module_platform_driver(imx_csi_driver);
+
+MODULE_DESCRIPTION("i.MX CSI subdev driver");
+MODULE_ALIAS("platform:imx-ipuv3-csi");
--
2.7.4
Sensor Interface module.
drivers/staging/media/imx/Kconfig | 14 +
drivers/staging/media/imx/Makefile | 2 +
drivers/staging/media/imx/imx-media-csi.c | 1311 +++++++++++++++++++++++++++++
3 files changed, 1327 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-csi.c
diff --git a/drivers/staging/media/imx/Kconfig b/drivers/staging/media/imx/Kconfig
index 62a3c34..e27ad6d 100644
--- a/drivers/staging/media/imx/Kconfig
+++ b/drivers/staging/media/imx/Kconfig
@@ -4,3 +4,17 @@ config VIDEO_IMX_MEDIA
---help---
Say yes here to enable support for video4linux media controller
driver for the i.MX5/6 SOC.
+
+if VIDEO_IMX_MEDIA
+menu "i.MX5/6 Media Sub devices"
+
+config VIDEO_IMX_CSI
+ tristate "i.MX5/6 Camera Sensor Interface driver"
+ depends on VIDEO_IMX_MEDIA && VIDEO_DEV && I2C
+ select VIDEOBUF2_DMA_CONTIG
+ default y
+ ---help---
+ A video4linux camera sensor interface driver for i.MX5/6.
+
+endmenu
+endif
diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile
index 4606a3a..c054490 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,3 +4,5 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
+
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
new file mode 100644
index 0000000..ab78ff7
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -0,0 +1,1311 @@
+/*
+ * V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ */
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-of.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-dma-contig.h>
+#include <video/imx-ipu-v3.h>
+#include <media/imx.h>
+#include "imx-media.h"
+
+#include <media/videobuf2-dma-contig.h>
+#include <video/imx-ipu-v3.h>
+#include <media/imx.h>
+#include "imx-media.h"
+
+/*
+ * Min/Max supported width and heights.
+ *
+ * Min/Max supported width and heights.
+ * We allow planar output, so we have to align width by 16 pixels
+ * to meet IDMAC alignment requirements.
+ *
+ * TODO: move this into pad format negotiation, if capture device
+ * has not requested planar formats, we should allow 8 pixel
+ * TODO: move this into pad format negotiation, if capture device
+ * alignment.
+ */
+#define MIN_W 176
+#define MIN_H 144
+#define MIN_W 176
+#define MIN_H 144
+#define MAX_W 4096
+#define MAX_H 4096
+#define MAX_H 4096
+#define W_ALIGN 4 /* multiple of 16 pixels */
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+struct csi_priv {
+#define H_ALIGN 1 /* multiple of 2 lines */
+#define S_ALIGN 1 /* multiple of 2 */
+
+ struct device *dev;
+ struct ipu_soc *ipu;
+ struct imx_media_dev *md;
+ struct v4l2_subdev sd;
+ struct media_pad pad[CSI_NUM_PADS];
+ /* the video device at IDMAC output pad */
+ struct imx_media_video_dev *vdev;
+ struct imx_media_fim *fim;
+ int csi_id;
+ int smfc_id;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ int active_output_pad;
+ /* lock to protect all members below */
+ struct mutex lock;
+
+
+ struct ipuv3_channel *idmac_ch;
+ struct ipu_smfc *smfc;
+ struct ipu_csi *csi;
+
+ struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
+ const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
+ struct v4l2_rect crop;
+
+
+ /* active vb2 buffers to send to video dev sink */
+ struct imx_media_buffer *active_vb2_buf[2];
+ struct imx_media_dma_buf underrun_buf;
+
+ int ipu_buf_num; /* ipu double buffer index: 0-1 */
+ struct imx_media_buffer *active_vb2_buf[2];
+ struct imx_media_dma_buf underrun_buf;
+
+
+ /* the sink for the captured frames */
+ struct media_entity *sink;
+ enum ipu_csi_dest dest;
+ struct media_entity *sink;
+ /* the source subdev */
+ struct v4l2_subdev *src_sd;
+
+ /* the mipi virtual channel number at link validate */
+
+ int vc_num;
+
+ /* the attached sensor at stream on */
+ struct imx_media_subdev *sensor;
+
+
+ spinlock_t irqlock; /* protect eof_irq handler */
+ struct timer_list eof_timeout_timer;
+ int eof_irq;
+ int nfb4eof_irq;
+
+ struct timer_list eof_timeout_timer;
+ int eof_irq;
+ int nfb4eof_irq;
+
+ struct v4l2_ctrl_handler ctrl_hdlr;
+
+ int power_count; /* power counter */
+
+ bool stream_on; /* streaming is on */
+ bool last_eof; /* waiting for last EOF at stream off */
+ struct completion last_eof_comp;
+};
+ bool last_eof; /* waiting for last EOF at stream off */
+ struct completion last_eof_comp;
+
+static inline struct csi_priv *sd_to_dev(struct v4l2_subdev *sdev)
+{
+ return container_of(sdev, struct csi_priv, sd);
+}
+
+static void csi_idmac_put_ipu_resources(struct csi_priv *priv)
+{
+ if (!IS_ERR_OR_NULL(priv->idmac_ch))
+ ipu_idmac_put(priv->idmac_ch);
+ priv->idmac_ch = NULL;
+
+ if (!IS_ERR_OR_NULL(priv->smfc))
+ ipu_smfc_put(priv->smfc);
+ priv->smfc = NULL;
+}
+
+static int csi_idmac_get_ipu_resources(struct csi_priv *priv)
+{
+ int ch_num, ret;
+
+ ch_num = IPUV3_CHANNEL_CSI0 + priv->smfc_id;
+
+ priv->smfc = ipu_smfc_get(priv->ipu, ch_num);
+ if (IS_ERR(priv->smfc)) {
+ v4l2_err(&priv->sd, "failed to get SMFC\n");
+ ret = PTR_ERR(priv->smfc);
+ goto out;
+ }
+
+ priv->idmac_ch = ipu_idmac_get(priv->ipu, ch_num);
+ }
+
+ if (IS_ERR(priv->idmac_ch)) {
+ v4l2_err(&priv->sd, "could not get IDMAC channel %u\n",
+ ch_num);
+ ret = PTR_ERR(priv->idmac_ch);
+ goto out;
+ }
+
+ return 0;
+out:
+
+ return 0;
+ csi_idmac_put_ipu_resources(priv);
+ return ret;
+}
+
+static void csi_vb2_buf_done(struct csi_priv *priv)
+}
+
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *done, *next;
+ struct vb2_buffer *vb;
+ dma_addr_t phys;
+
+ done = priv->active_vb2_buf[priv->ipu_buf_num];
+ if (done) {
+ vb = &done->vbuf.vb2_buf;
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+ }
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *done, *next;
+ struct vb2_buffer *vb;
+ dma_addr_t phys;
+
+ done = priv->active_vb2_buf[priv->ipu_buf_num];
+ if (done) {
+ vb = &done->vbuf.vb2_buf;
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+ }
+
+ /* get next queued buffer */
+ /* get next queued buffer */
+ next = imx_media_capture_device_next_buf(vdev);
+ if (next) {
+ phys = vb2_dma_contig_plane_dma_addr(&next->vbuf.vb2_buf, 0);
+ priv->active_vb2_buf[priv->ipu_buf_num] = next;
+ } else {
+ phys = priv->underrun_buf.phys;
+ priv->active_vb2_buf[priv->ipu_buf_num] = NULL;
+ if (next) {
+ phys = vb2_dma_contig_plane_dma_addr(&next->vbuf.vb2_buf, 0);
+ priv->active_vb2_buf[priv->ipu_buf_num] = next;
+ } else {
+ phys = priv->underrun_buf.phys;
+ }
+
+ if (ipu_idmac_buffer_is_ready(priv->idmac_ch, priv->ipu_buf_num))
+ ipu_idmac_clear_buffer(priv->idmac_ch, priv->ipu_buf_num);
+
+ ipu_cpmem_set_buffer(priv->idmac_ch, priv->ipu_buf_num, phys);
+}
+
+static void csi_call_fim(struct csi_priv *priv)
+{
+ if (priv->fim) {
+ struct timespec cur_ts;
+
+ ktime_get_ts(&cur_ts);
+ /* call frame interval monitor */
+ imx_media_fim_eof_monitor(priv->fim, &cur_ts);
+ }
+}
+
+static irqreturn_t csi_idmac_eof_interrupt(int irq, void *dev_id)
+{
+ struct csi_priv *priv = dev_id;
+
+ spin_lock(&priv->irqlock);
+
+ if (priv->last_eof) {
+ complete(&priv->last_eof_comp);
+ priv->last_eof = false;
+ goto unlock;
+ }
+
+ csi_call_fim(priv);
+ spin_lock(&priv->irqlock);
+
+ if (priv->last_eof) {
+ complete(&priv->last_eof_comp);
+ priv->last_eof = false;
+ goto unlock;
+ }
+
+
+ csi_vb2_buf_done(priv);
+
+ /* select new IPU buf */
+ ipu_idmac_select_buffer(priv->idmac_ch, priv->ipu_buf_num);
+ /* select new IPU buf */
+ /* toggle IPU double-buffer index */
+ priv->ipu_buf_num ^= 1;
+
+ /* bump the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+unlock:
+ spin_unlock(&priv->irqlock);
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t csi_idmac_nfb4eof_interrupt(int irq, void *dev_id)
+ priv->ipu_buf_num ^= 1;
+
+ /* bump the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+unlock:
+ spin_unlock(&priv->irqlock);
+ return IRQ_HANDLED;
+}
+
+{
+ struct csi_priv *priv = dev_id;
+ static const struct v4l2_event ev = {
+ .type = V4L2_EVENT_NEW_FRAME_BEFORE_EOF,
+ };
+
+ v4l2_err(&priv->sd, "NFB4EOF\n");
+ .type = V4L2_EVENT_NEW_FRAME_BEFORE_EOF,
+ };
+
+
+ v4l2_subdev_notify_event(&priv->sd, &ev);
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * EOF timeout timer function. This is an unrecoverable condition
+ * without a stream restart.
+ */
+static void csi_idmac_eof_timeout(unsigned long data)
+ * without a stream restart.
+ */
+{
+ struct csi_priv *priv = (struct csi_priv *)data;
+ struct imx_media_video_dev *vdev = priv->vdev;
+
+ v4l2_err(&priv->sd, "EOF timeout\n");
+
+ /* signal a fatal error to capture device */
+ imx_media_capture_device_error(vdev);
+}
+
+static void csi_idmac_setup_vb2_buf(struct csi_priv *priv, dma_addr_t *phys)
+ /* signal a fatal error to capture device */
+ imx_media_capture_device_error(vdev);
+}
+
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *buf;
+ int i;
+
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct imx_media_buffer *buf;
+ int i;
+ for (i = 0; i < 2; i++) {
+ buf = imx_media_capture_device_next_buf(vdev);
+ priv->active_vb2_buf[i] = buf;
+ phys[i] = vb2_dma_contig_plane_dma_addr(&buf->vbuf.vb2_buf, 0);
+ }
+ buf = imx_media_capture_device_next_buf(vdev);
+ priv->active_vb2_buf[i] = buf;
+ phys[i] = vb2_dma_contig_plane_dma_addr(&buf->vbuf.vb2_buf, 0);
+}
+
+static void csi_idmac_unsetup_vb2_buf(struct csi_priv *priv,
+ enum vb2_buffer_state return_state)
+{
+ struct imx_media_buffer *buf;
+ int i;
+
+ /* return any remaining active frames with error */
+ for (i = 0; i < 2; i++) {
+ buf = priv->active_vb2_buf[i];
+ if (buf) {
+ struct vb2_buffer *vb = &buf->vbuf.vb2_buf;
+
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, return_state);
+ }
+ }
+}
+
+/* init the SMFC IDMAC channel */
+ for (i = 0; i < 2; i++) {
+ buf = priv->active_vb2_buf[i];
+ if (buf) {
+ struct vb2_buffer *vb = &buf->vbuf.vb2_buf;
+
+ vb->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb, return_state);
+ }
+ }
+}
+
+static int csi_idmac_setup_channel(struct csi_priv *priv)
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_of_endpoint *sensor_ep;
+ struct v4l2_mbus_framefmt *infmt;
+ unsigned int burst_size;
+ struct ipu_image image;
+ dma_addr_t phys[2];
+ struct ipu_image image;
+ bool passthrough;
+ int ret;
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ ipu_cpmem_zero(priv->idmac_ch);
+
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ memset(&image, 0, sizeof(image));
+ image.pix = vdev->fmt.fmt.pix;
+ image.rect.width = image.pix.width;
+ image.rect.height = image.pix.height;
+
+ csi_idmac_setup_vb2_buf(priv, phys);
+ image.rect.height = image.pix.height;
+
+
+ image.phys0 = phys[0];
+ image.phys1 = phys[1];
+
+ ret = ipu_cpmem_set_image(priv->idmac_ch, &image);
+ if (ret)
+ return ret;
+
+ return ret;
+
+ burst_size = (image.pix.width & 0xf) ? 8 : 16;
+
+ ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
+
+
+ /*
+ * If the sensor uses 16-bit parallel CSI bus, we must handle
+ * the data internally in the IPU as 16-bit generic, aka
+ * passthrough mode.
+ */
+ passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+ sensor_ep->bus.parallel.bus_width >= 16);
+
+ if (passthrough)
+ sensor_ep->bus.parallel.bus_width >= 16);
+
+ ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+
+ /*
+ * Set the channel for the direct CSI-->memory via SMFC
+ * use-case to very high priority, by enabling the watermark
+ * signal in the SMFC, enabling WM in the channel, and setting
+ * the channel priority to high.
+ *
+ * Refer to the i.mx6 rev. D TRM Table 36-8: Calculated priority
+ * value.
+ *
+ * The WM's are set very low by intention here to ensure that
+ * the SMFC FIFOs do not overflow.
+ */
+ ipu_smfc_set_watermark(priv->smfc, 0x02, 0x01);
+ ipu_cpmem_set_high_priority(priv->idmac_ch);
+ ipu_idmac_enable_watermark(priv->idmac_ch, true);
+ ipu_cpmem_set_axi_id(priv->idmac_ch, 0);
+ ipu_idmac_lock_enable(priv->idmac_ch, 8);
+
+ burst_size = ipu_cpmem_get_burstsize(priv->idmac_ch);
+ burst_size = passthrough ?
+ (burst_size >> 3) - 1 : (burst_size >> 2) - 1;
+
+ ipu_smfc_set_burstsize(priv->smfc, burst_size);
+
+ if (image.pix.field == V4L2_FIELD_NONE &&
+ V4L2_FIELD_HAS_BOTH(infmt->field))
+ if (image.pix.field == V4L2_FIELD_NONE &&
+ ipu_cpmem_interlaced_scan(priv->idmac_ch,
+ image.pix.bytesperline);
+
+ ipu_idmac_set_double_buffer(priv->idmac_ch, true);
+
+ return 0;
+}
+
+static void csi_idmac_unsetup(struct csi_priv *priv,
+ return 0;
+}
+
+ enum vb2_buffer_state state)
+{
+ ipu_idmac_disable_channel(priv->idmac_ch);
+{
+ ipu_smfc_disable(priv->smfc);
+
+ csi_idmac_unsetup_vb2_buf(priv, state);
+}
+
+static int csi_idmac_setup(struct csi_priv *priv)
+{
+ int ret;
+
+ ret = csi_idmac_setup_channel(priv);
+ int ret;
+
+ if (ret)
+ return ret;
+
+ ipu_cpmem_dump(priv->idmac_ch);
+ return ret;
+
+ ipu_dump(priv->ipu);
+
+ ipu_smfc_enable(priv->smfc);
+
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->idmac_ch, 0);
+ /* set buffers ready */
+ ipu_idmac_select_buffer(priv->idmac_ch, 1);
+
+ /* enable the channels */
+ ipu_idmac_enable_channel(priv->idmac_ch);
+ /* enable the channels */
+
+ return 0;
+}
+
+static int csi_idmac_start(struct csi_priv *priv)
+ return 0;
+}
+
+{
+ struct imx_media_video_dev *vdev = priv->vdev;
+ struct v4l2_pix_format *outfmt;
+ int ret;
+
+ ret = csi_idmac_get_ipu_resources(priv);
+ if (ret)
+ return ret;
+
+ ipu_smfc_map_channel(priv->smfc, priv->csi_id, priv->vc_num);
+ return ret;
+
+
+ outfmt = &vdev->fmt.fmt.pix;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->underrun_buf,
+ outfmt->sizeimage);
+ outfmt = &vdev->fmt.fmt.pix;
+
+ ret = imx_media_alloc_dma_buf(priv->md, &priv->underrun_buf,
+ outfmt->sizeimage);
+ if (ret)
+ goto out_put_ipu;
+
+ goto out_put_ipu;
+
+ priv->ipu_buf_num = 0;
+
+ /* init EOF completion waitq */
+ init_completion(&priv->last_eof_comp);
+ priv->last_eof = false;
+
+ ret = csi_idmac_setup(priv);
+
+ /* init EOF completion waitq */
+ init_completion(&priv->last_eof_comp);
+ priv->last_eof = false;
+
+ if (ret) {
+ v4l2_err(&priv->sd, "csi_idmac_setup failed: %d\n", ret);
+ goto out_free_dma_buf;
+ }
+
+ priv->nfb4eof_irq = ipu_idmac_channel_irq(priv->ipu,
+ priv->idmac_ch,
+ priv->nfb4eof_irq = ipu_idmac_channel_irq(priv->ipu,
+ IPU_IRQ_NFB4EOF);
+ ret = devm_request_irq(priv->dev, priv->nfb4eof_irq,
+ csi_idmac_nfb4eof_interrupt, 0,
+ "imx-smfc-nfb4eof", priv);
+ if (ret) {
+ v4l2_err(&priv->sd,
+ "Error registering NFB4EOF irq: %d\n", ret);
+ goto out_unsetup;
+ }
+
+ priv->eof_irq = ipu_idmac_channel_irq(priv->ipu, priv->idmac_ch,
+ goto out_unsetup;
+ }
+
+ IPU_IRQ_EOF);
+
+ ret = devm_request_irq(priv->dev, priv->eof_irq,
+ csi_idmac_eof_interrupt, 0,
+ "imx-smfc-eof", priv);
+ if (ret) {
+ v4l2_err(&priv->sd,
+ "Error registering eof irq: %d\n", ret);
+ goto out_free_nfb4eof_irq;
+ }
+
+ /* start the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+ goto out_free_nfb4eof_irq;
+ }
+
+ /* start the EOF timeout timer */
+ mod_timer(&priv->eof_timeout_timer,
+ jiffies + msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+
+ return 0;
+
+out_free_nfb4eof_irq:
+ return 0;
+
+ devm_free_irq(priv->dev, priv->nfb4eof_irq, priv);
+out_unsetup:
+ csi_idmac_unsetup(priv, VB2_BUF_STATE_QUEUED);
+out_free_dma_buf:
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+out_put_ipu:
+ csi_idmac_put_ipu_resources(priv);
+out_put_ipu:
+ return ret;
+}
+
+static void csi_idmac_stop(struct csi_priv *priv)
+}
+
+{
+ unsigned long flags;
+ int ret;
+
+ /* mark next EOF interrupt as the last before stream off */
+ spin_lock_irqsave(&priv->irqlock, flags);
+ priv->last_eof = true;
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+
+ spin_lock_irqsave(&priv->irqlock, flags);
+ priv->last_eof = true;
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+ /*
+ * and then wait for interrupt handler to mark completion.
+ */
+ ret = wait_for_completion_timeout(
+ &priv->last_eof_comp, msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+ */
+ ret = wait_for_completion_timeout(
+ if (ret == 0)
+ v4l2_warn(&priv->sd, "wait last EOF timeout\n");
+
+ devm_free_irq(priv->dev, priv->eof_irq, priv);
+ devm_free_irq(priv->dev, priv->nfb4eof_irq, priv);
+
+ csi_idmac_unsetup(priv, VB2_BUF_STATE_ERROR);
+
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+
+ /* cancel the EOF timeout timer */
+ del_timer_sync(&priv->eof_timeout_timer);
+
+ csi_idmac_put_ipu_resources(priv);
+ imx_media_free_dma_buf(priv->md, &priv->underrun_buf);
+
+ /* cancel the EOF timeout timer */
+ del_timer_sync(&priv->eof_timeout_timer);
+
+}
+
+/* Update the CSI whole sensor and active windows */
+static int csi_setup(struct csi_priv *priv)
+{
+ struct v4l2_mbus_framefmt *infmt, *outfmt;
+ struct v4l2_mbus_config sensor_mbus_cfg;
+ struct v4l2_of_endpoint *sensor_ep;
+ struct v4l2_mbus_framefmt if_fmt;
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+ outfmt = &priv->format_mbus[priv->active_output_pad];
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ /* compose mbus_config from sensor endpoint */
+ sensor_mbus_cfg.type = sensor_ep->bus_type;
+ sensor_mbus_cfg.flags = (sensor_ep->bus_type == V4L2_MBUS_CSI2) ?
+ sensor_ep->bus.mipi_csi2.flags :
+ sensor_ep->bus.parallel.flags;
+
+ /*
+ * we need to pass input sensor frame to CSI interface, but
+ * with translated field type from output format
+ */
+ if_fmt = *infmt;
+ if_fmt.field = outfmt->field;
+
+ ipu_csi_set_window(priv->csi, &priv->crop);
+
+ ipu_csi_init_interface(priv->csi, &sensor_mbus_cfg, &if_fmt);
+
+ ipu_csi_set_dest(priv->csi, priv->dest);
+
+ ipu_csi_dump(priv->csi);
+
+ return 0;
+}
+
+static int csi_start(struct csi_priv *priv)
+ return 0;
+}
+
+{
+ int ret;
+
+ if (!priv->sensor) {
+ int ret;
+
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return -EINVAL;
+ }
+
+ if (priv->dest == IPU_CSI_DEST_IDMAC) {
+ ret = csi_idmac_start(priv);
+ if (ret)
+ return ret;
+ }
+
+ ret = csi_setup(priv);
+ return ret;
+ }
+
+ if (ret)
+ goto idmac_stop;
+
+ /* start the frame interval monitor */
+ if (priv->fim) {
+ ret = imx_media_fim_set_stream(priv->fim, priv->sensor, true);
+ if (ret)
+ goto idmac_stop;
+ }
+
+ ret = ipu_csi_enable(priv->csi);
+ if (ret) {
+ v4l2_err(&priv->sd, "CSI enable error: %d\n", ret);
+ goto fim_off;
+ }
+
+ return 0;
+
+fim_off:
+
+ return 0;
+
+ if (priv->fim)
+ imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+idmac_stop:
+ if (priv->dest == IPU_CSI_DEST_IDMAC)
+ csi_idmac_stop(priv);
+ return ret;
+}
+
+static void csi_stop(struct csi_priv *priv)
+}
+
+{
+ if (priv->dest == IPU_CSI_DEST_IDMAC)
+ csi_idmac_stop(priv);
+
+ /* stop the frame interval monitor */
+ if (priv->fim)
+ imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+
+ ipu_csi_disable(priv->csi);
+}
+
+/*
+ * V4L2 subdev operations.
+ */
+
+static int csi_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+
+
+ if (!priv->src_sd || !priv->sink) {
+ ret = -EPIPE;
+ ret = -EPIPE;
+ goto out;
+ }
+
+ }
+
+ dev_dbg(priv->dev, "stream %s\n", enable ? "ON" : "OFF");
+
+ if (enable && !priv->stream_on)
+ ret = csi_start(priv);
+
+ if (enable && !priv->stream_on)
+ else if (!enable && priv->stream_on)
+ csi_stop(priv);
+
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ if (!ret)
+ priv->stream_on = enable;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_s_power(struct v4l2_subdev *sd, int on)
+}
+
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ int ret = 0;
+
+ dev_dbg(priv->dev, "power %s\n", on ? "ON" : "OFF");
+
+
+ mutex_lock(&priv->lock);
+
+ /*
+ * If the power count is modified from 0 to != 0 or from != 0 to 0,
+ * update the power state.
+ */
+ if (priv->power_count == !on) {
+ * update the power state.
+ */
+ if (priv->fim) {
+ ret = imx_media_fim_set_power(priv->fim, on);
+ if (ret)
+ goto out;
+ }
+ }
+
+ }
+ }
+
+ /* Update the power count. */
+ priv->power_count += on ? 1 : -1;
+ WARN_ON(priv->power_count < 0);
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_link_setup(struct media_entity *entity,
+}
+
+ const struct media_pad *local,
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ const struct media_pad *remote, u32 flags)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct v4l2_subdev *remote_sd;
+ int ret = 0;
+
+ dev_dbg(priv->dev, "link setup %s -> %s\n", remote->entity->name,
+
+ local->entity->name);
+
+ mutex_lock(&priv->lock);
+
+
+ if (local->flags & MEDIA_PAD_FL_SINK) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ goto out;
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->src_sd) {
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ ret = -EBUSY;
+ goto out;
+ }
+ goto out;
+ priv->src_sd = remote_sd;
+ } else {
+ priv->src_sd = NULL;
+ }
+
+ goto out;
+ }
+
+ /* this is a source pad */
+ }
+
+
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ if (priv->sink) {
+ if (flags & MEDIA_LNK_FL_ENABLED) {
+ ret = -EBUSY;
+ goto out;
+ }
+ goto out;
+ }
+ } else {
+ priv->sink = NULL;
+ priv->sink = NULL;
+ goto out;
+ }
+
+ /* record which output pad is now active */
+ }
+
+ priv->active_output_pad = local->index;
+
+ /* set CSI destination */
+ if (local->index == CSI_SRC_PAD_IDMAC) {
+ if (!is_media_entity_v4l2_video_device(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ goto out;
+ }
+ priv->dest = IPU_CSI_DEST_IDMAC;
+ } else {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ if (!is_media_entity_v4l2_subdev(remote->entity)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ goto out;
+ }
+
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+ switch (remote_sd->grp_id) {
+ remote_sd = media_entity_to_v4l2_subdev(remote->entity);
+ case IMX_MEDIA_GRP_ID_VDIC:
+ priv->dest = IPU_CSI_DEST_VDIC;
+ break;
+ case IMX_MEDIA_GRP_ID_IC_PRP:
+ priv->dest = IPU_CSI_DEST_IC;
+ break;
+ default:
+ ret = -EINVAL;
+ goto out;
+ }
+ goto out;
+ }
+ }
+
+ priv->sink = remote->entity;
+out:
+ mutex_unlock(&priv->lock);
+
+ priv->sink = remote->entity;
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_link_validate(struct v4l2_subdev *sd,
+}
+
+ struct media_link *link,
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *source_fmt,
+ struct v4l2_subdev_format *sink_fmt)
+{
+ struct v4l2_of_endpoint *sensor_ep;
+ struct imx_media_subdev *sensor;
+ bool is_csi2;
+ int ret;
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ ret = v4l2_subdev_link_validate_frame_interval(link);
+
+ ret = v4l2_subdev_link_validate_default(sd, link,
+ source_fmt, sink_fmt);
+ if (ret)
+ return ret;
+
+ if (ret)
+ return ret;
+
+ sensor = __imx_media_find_sensor(priv->md, &priv->sd.entity);
+ return ret;
+
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(priv->sensor);
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ }
+
+ mutex_lock(&priv->lock);
+
+ priv->sensor = sensor;
+
+ mutex_lock(&priv->lock);
+
+ sensor_ep = &priv->sensor->sensor_ep;
+
+ is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+
+ if (is_csi2) {
+ int vc_num = 0;
+ /*
+ * NOTE! It seems the virtual channels from the mipi csi-2
+ * receiver are used only for routing by the video mux's,
+ * or for hard-wired routing to the CSI's. Once the stream
+ * enters the CSI's however, they are treated internally
+ * in the IPU as virtual channel 0.
+ */
+#if 0
+ mutex_unlock(&priv->lock);
+ vc_num = imx_media_find_mipi_csi2_channel(priv->md,
+ &priv->sd.entity);
+ if (vc_num < 0)
+ return vc_num;
+ mutex_lock(&priv->lock);
+#endif
+ ipu_csi_set_mipi_datatype(priv->csi, vc_num,
+ &priv->format_mbus[CSI_SINK_PAD]);
+ }
+
+ /* select either parallel or MIPI-CSI2 as input to CSI */
+ ipu_set_csi_src_mux(priv->ipu, priv->csi_id, is_csi2);
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_eof_isr(struct v4l2_subdev *sd, u32 status, bool *handled)
+}
+
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->irqlock, flags);
+
+ csi_call_fim(priv);
+ spin_unlock_irqrestore(&priv->irqlock, flags);
+
+ return 0;
+}
+
+static int csi_try_crop(struct csi_priv *priv, struct v4l2_rect *crop,
+ return 0;
+}
+
+ struct imx_media_subdev *sensor)
+{
+ struct v4l2_of_endpoint *sensor_ep;
+ struct v4l2_mbus_framefmt *infmt;
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+
+ sensor_ep = &sensor->sensor_ep;
+
+ crop->width = min_t(__u32, infmt->width, crop->width);
+ if (crop->left + crop->width > infmt->width)
+ crop->left = infmt->width - crop->width;
+ /* adjust crop left/width to h/w alignment restrictions */
+ crop->left &= ~0x3;
+ crop->width &= ~0x7;
+
+ /*
+ * FIXME: not sure why yet, but on interlaced bt.656,
+ * changing the vertical cropping causes loss of vertical
+ * sync, so fix it to NTSC/PAL active lines. NTSC contains
+ * 2 extra lines of active video that need to be cropped.
+ */
+ if (sensor_ep->bus_type == V4L2_MBUS_BT656 &&
+ (V4L2_FIELD_HAS_BOTH(infmt->field) ||
+ infmt->field == V4L2_FIELD_ALTERNATE)) {
+ crop->height = infmt->height;
+ crop->top = (infmt->height == 480) ? 2 : 0;
+ } else {
+ crop->height = min_t(__u32, infmt->height, crop->height);
+ if (crop->top + crop->height > infmt->height)
+ crop->top = infmt->height - crop->height;
+ }
+
+ return 0;
+}
+
+static int csi_enum_mbus_code(struct v4l2_subdev *sd,
+
+ return 0;
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= CSI_NUM_PADS)
+{
+ return -EINVAL;
+
+ if (code->pad == CSI_SRC_PAD_DIRECT)
+ return imx_media_enum_ipu_format(NULL, &code->code,
+ code->index, true);
+
+ return imx_media_enum_format(NULL, &code->code, code->index,
+ true, false);
+}
+
+static int csi_get_fmt(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+
+ if (sdformat->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ sdformat->format = priv->format_mbus[sdformat->pad];
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int csi_set_fmt(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_subdev_format *sdformat)
+{
+ const struct imx_media_pixfmt *cc, *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ struct imx_media_subdev *sensor;
+ struct v4l2_rect crop;
+ int ret = 0;
+ u32 code;
+
+ if (sdformat->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
+ sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+ }
+
+
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+ switch (sdformat->pad) {
+ case CSI_SRC_PAD_DIRECT:
+ switch (sdformat->pad) {
+ case CSI_SRC_PAD_IDMAC:
+ crop.left = priv->crop.left;
+ crop.top = priv->crop.top;
+ crop.width = sdformat->format.width;
+ crop.height = sdformat->format.height;
+ ret = csi_try_crop(priv, &crop, sensor);
+ if (ret)
+ crop.height = sdformat->format.height;
+ ret = csi_try_crop(priv, &crop, sensor);
+ goto out;
+ sdformat->format.width = crop.width;
+ sdformat->format.height = crop.height;
+
+ if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
+ cc = imx_media_find_format(0, sdformat->format.code,
+ true, false);
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0,
+ true, false);
+ imx_media_enum_format(NULL, &code, 0,
+ cc = imx_media_find_format(0, code,
+ true, false);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ incc = priv->cc[CSI_SINK_PAD];
+ if (cc->cs != incc->cs) {
+ sdformat->format.code = infmt->code;
+ cc = imx_media_find_format(
+ 0, sdformat->format.code,
+ true, false);
+ }
+
+ }
+
+ if (sdformat->format.field != V4L2_FIELD_NONE)
+ sdformat->format.field = infmt->field;
+ } else {
+ sdformat->format.field = infmt->field;
+ cc = imx_media_find_ipu_format(0, sdformat->format.code,
+ true);
+ if (!cc) {
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+ cc = imx_media_find_ipu_format(0, code, true);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ sdformat->format.field = infmt->field;
+ }
+
+ /*
+ * translate V4L2_FIELD_ALTERNATE to SEQ_TB or SEQ_BT
+ * depending on input height (assume NTSC top-bottom
+ * order if 480 lines, otherwise PAL bottom-top order).
+ */
+ if (sdformat->format.field == V4L2_FIELD_ALTERNATE) {
+ sdformat->format.field = (infmt->height == 480) ?
+ V4L2_FIELD_SEQ_TB : V4L2_FIELD_SEQ_BT;
+ }
+ break;
+ case CSI_SINK_PAD:
+ cc = imx_media_find_format(0, sdformat->format.code,
+ true, false);
+ if (!cc) {
+ imx_media_enum_format(NULL, &code, 0, true, false);
+ cc = imx_media_find_format(0, code, true, false);
+ sdformat->format.code = cc->codes[0];
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ goto out;
+ }
+
+ goto out;
+ }
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_fmt = sdformat->format;
+ } else {
+ priv->format_mbus[sdformat->pad] = sdformat->format;
+ priv->cc[sdformat->pad] = cc;
+ /* Update the crop window if this is an output pad */
+ priv->cc[sdformat->pad] = cc;
+ if (sdformat->pad == CSI_SRC_PAD_DIRECT ||
+ sdformat->pad == CSI_SRC_PAD_IDMAC)
+ priv->crop = crop;
+ }
+
+out:
+ }
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_get_selection(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+
+ if (sel->pad >= CSI_NUM_PADS || sel->pad == CSI_SINK_PAD)
+
+ return -EINVAL;
+
+ mutex_lock(&priv->lock);
+
+ infmt = &priv->format_mbus[CSI_SINK_PAD];
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ sel->r.left = 0;
+ sel->r.top = 0;
+ sel->r.width = infmt->width;
+ sel->r.height = infmt->height;
+ break;
+ case V4L2_SEL_TGT_CROP:
+ sel->r = priv->crop;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_set_selection(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *outfmt;
+ struct imx_media_subdev *sensor;
+ int ret = 0;
+
+ if (sel->pad >= CSI_NUM_PADS ||
+
+ sel->pad == CSI_SINK_PAD ||
+ sel->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ sensor = imx_media_find_sensor(priv->md, &priv->sd.entity);
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ goto out;
+ if (IS_ERR(sensor)) {
+ v4l2_err(&priv->sd, "no sensor attached\n");
+ return PTR_ERR(sensor);
+ }
+
+ mutex_lock(&priv->lock);
+
+ if (priv->stream_on) {
+ ret = -EBUSY;
+ }
+
+ /*
+ * Modifying the crop rectangle always changes the format on the source
+ * pad. If the KEEP_CONFIG flag is set, just return the current crop
+ * rectangle.
+ */
+ if (sel->flags & V4L2_SEL_FLAG_KEEP_CONFIG) {
+ sel->r = priv->crop;
+ if (sel->which == V4L2_SUBDEV_FORMAT_TRY)
+ cfg->try_crop = sel->r;
+ goto out;
+ }
+
+ outfmt = &priv->format_mbus[sel->pad];
+ }
+
+
+ ret = csi_try_crop(priv, &sel->r, sensor);
+ if (ret)
+ goto out;
+
+ if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
+ cfg->try_crop = sel->r;
+ } else {
+ priv->crop = sel->r;
+ /* Update the source format */
+ outfmt->width = sel->r.width;
+ outfmt->height = sel->r.height;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+static int csi_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+}
+
+ struct v4l2_event_subscription *sub)
+{
+ if (sub->type != V4L2_EVENT_NEW_FRAME_BEFORE_EOF &&
+{
+ sub->type != V4L2_EVENT_FRAME_INTERVAL_ERROR)
+ return -EINVAL;
+ if (sub->id != 0)
+ return -EINVAL;
+ if (sub->id != 0)
+
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+}
+
+static int csi_unsubscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+}
+
+ struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_unsubscribe(fh, sub);
+}
+{
+ return v4l2_event_unsubscribe(fh, sub);
+
+/*
+ * retrieve our pads parsed from the OF graph by the media device
+ */
+static int csi_registered(struct v4l2_subdev *sd)
+ */
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ int i, ret;
+ u32 code;
+ u32 code;
+
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ /* get handle to IPU CSI */
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+
+ priv->csi = ipu_csi_get(priv->ipu, priv->csi_id);
+ if (IS_ERR(priv->csi)) {
+ v4l2_err(&priv->sd, "failed to get CSI%d\n", priv->csi_id);
+ return PTR_ERR(priv->csi);
+ }
+
+ for (i = 0; i < CSI_NUM_PADS; i++) {
+ priv->pad[i].flags = (i == CSI_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ code = 0;
+ if (i == CSI_SRC_PAD_DIRECT)
+ code = 0;
+ imx_media_enum_ipu_format(NULL, &code, 0, true);
+
+
+ /* set a default mbus format */
+ ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ goto put_csi;
+ 640, 480, code, V4L2_FIELD_NONE,
+ &priv->cc[i]);
+ if (ret)
+ }
+
+ priv->fim = imx_media_fim_init(&priv->sd);
+ if (IS_ERR(priv->fim)) {
+ ret = PTR_ERR(priv->fim);
+ goto put_csi;
+ }
+
+ ret = media_entity_pads_init(&sd->entity, CSI_NUM_PADS, priv->pad);
+ if (ret)
+ goto free_fim;
+
+ ret = imx_media_capture_device_register(priv->vdev);
+ if (ret)
+ ret = imx_media_capture_device_register(priv->vdev);
+ goto free_fim;
+
+ return 0;
+
+free_fim:
+ return 0;
+
+ if (priv->fim)
+ imx_media_fim_free(priv->fim);
+put_csi:
+ ipu_csi_put(priv->csi);
+ return ret;
+}
+
+static void csi_unregistered(struct v4l2_subdev *sd)
+}
+
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+ imx_media_capture_device_unregister(priv->vdev);
+
+ if (priv->fim)
+ imx_media_fim_free(priv->fim);
+
+ if (!IS_ERR_OR_NULL(priv->csi))
+ ipu_csi_put(priv->csi);
+}
+
+static struct media_entity_operations csi_entity_ops = {
+ .link_setup = csi_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static struct v4l2_subdev_core_ops csi_core_ops = {
+};
+
+ .s_power = csi_s_power,
+ .interrupt_service_routine = csi_eof_isr,
+ .subscribe_event = csi_subscribe_event,
+ .unsubscribe_event = csi_unsubscribe_event,
+};
+
+static struct v4l2_subdev_video_ops csi_video_ops = {
+ .s_stream = csi_s_stream,
+};
+
+static struct v4l2_subdev_pad_ops csi_pad_ops = {
+ .enum_mbus_code = csi_enum_mbus_code,
+ .get_fmt = csi_get_fmt,
+ .set_fmt = csi_set_fmt,
+ .get_selection = csi_get_selection,
+ .set_selection = csi_set_selection,
+ .link_validate = csi_link_validate,
+};
+
+static struct v4l2_subdev_ops csi_subdev_ops = {
+ .core = &csi_core_ops,
+ .video = &csi_video_ops,
+ .pad = &csi_pad_ops,
+};
+
+static struct v4l2_subdev_internal_ops csi_internal_ops = {
+ .registered = csi_registered,
+ .unregistered = csi_unregistered,
+};
+
+static int imx_csi_probe(struct platform_device *pdev)
+{
+ struct ipu_client_platformdata *pdata;
+ struct pinctrl *pinctrl;
+ struct csi_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, &priv->sd);
+ priv->dev = &pdev->dev;
+
+ ret = dma_set_coherent_mask(priv->dev, DMA_BIT_MASK(32));
+ if (ret)
+ return ret;
+
+ /* get parent IPU */
+ return ret;
+
+ priv->ipu = dev_get_drvdata(priv->dev->parent);
+
+ /* get our CSI id */
+ pdata = priv->dev->platform_data;
+ priv->csi_id = pdata->csi;
+ priv->smfc_id = (priv->csi_id == 0) ? 0 : 2;
+
+ init_timer(&priv->eof_timeout_timer);
+ priv->eof_timeout_timer.data = (unsigned long)priv;
+ priv->eof_timeout_timer.function = csi_idmac_eof_timeout;
+ priv->eof_timeout_timer.data = (unsigned long)priv;
+ spin_lock_init(&priv->irqlock);
+
+ v4l2_subdev_init(&priv->sd, &csi_subdev_ops);
+ v4l2_set_subdevdata(&priv->sd, priv);
+ priv->sd.internal_ops = &csi_internal_ops;
+ priv->sd.entity.ops = &csi_entity_ops;
+ priv->sd.of_node = pdata->of_node;
+ priv->sd.owner = THIS_MODULE;
+ priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ priv->sd.grp_id = priv->csi_id ?
+ IMX_MEDIA_GRP_ID_CSI1 : IMX_MEDIA_GRP_ID_CSI0;
+ imx_media_grp_id_to_sd_name(priv->sd.name, sizeof(priv->sd.name),
+ priv->sd.grp_id, ipu_get_num(priv->ipu));
+
+ priv->vdev = imx_media_capture_device_init(&priv->sd,
+ CSI_SRC_PAD_IDMAC);
+ if (IS_ERR(priv->vdev))
+ return PTR_ERR(priv->vdev);
+ return PTR_ERR(priv->vdev);
+
+ v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
+ priv->sd.ctrl_handler = &priv->ctrl_hdlr;
+ v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
+
+ /*
+ * The IPUv3 driver did not assign an of_node to this
+ * device. As a result, pinctrl does not automatically
+ * configure our pin groups, so we need to do that manually
+ * here, after setting this device's of_node.
+ */
+ priv->dev->of_node = pdata->of_node;
+ pinctrl = devm_pinctrl_get_select_default(priv->dev);
+
+ mutex_init(&priv->lock);
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ goto free;
+
+ return 0;
+free:
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+ mutex_init(&priv->lock);
+
+ ret = v4l2_async_register_subdev(&priv->sd);
+ if (ret)
+ goto free;
+
+ return 0;
+free:
+ mutex_destroy(&priv->lock);
+ imx_media_capture_device_remove(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
+ return ret;
+}
+
+static int imx_csi_remove(struct platform_device *pdev)
+}
+
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct csi_priv *priv = sd_to_dev(sd);
+
+ v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
+ mutex_destroy(&priv->lock);
+ imx_media_capture_device_remove(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
+ v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+static const struct platform_device_id imx_csi_ids[] = {
+ media_entity_cleanup(&sd->entity);
+
+ return 0;
+}
+
+ { .name = "imx-ipuv3-csi" },
+ { },
+};
+MODULE_DEVICE_TABLE(platform, imx_csi_ids);
+
+static struct platform_driver imx_csi_driver = {
+ .probe = imx_csi_probe,
+ .remove = imx_csi_remove,
+ .id_table = imx_csi_ids,
+ .driver = {
+ .name = "imx-ipuv3-csi",
+ },
+};
+module_platform_driver(imx_csi_driver);
+
+MODULE_DESCRIPTION("i.MX CSI subdev driver");
+MODULE_ALIAS("platform:imx-ipuv3-csi");
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
The previous API and negotiation of mbus codes and pixel formats
was broken, and has been completely redone.
The negotiation of media bus codes should be as follows:
CSI:
sink pad direct src pad IDMAC src pad
-------- ---------------- -------------
RGB (any) IPU RGB IPU RGB
YUV (any) IPU YUV IPU YUV
Bayer N/A must be same bayer code as sink
VDIC:
direct sink pad IDMAC sink pad direct src pad
--------------- -------------- --------------
IPU YUV only IPU YUV only IPU YUV only
PRP:
direct sink pad direct src pads
--------------- ---------------
IPU (any) same as sink code
PRP ENC/VF:
direct sink pad IDMAC src pads
--------------- --------------
IPU (any) IPU RGB or IPU YUV
Given the above, a new internal API is created:
enum codespace_sel {
CS_SEL_YUV = 0, /* find or enumerate only YUV codes */
CS_SEL_RGB, /* find or enumerate only RGB codes */
CS_SEL_ANY, /* find or enumerate both YUV and RGB codes */
};
/* Find and enumerate fourcc pixel formats */
const struct imx_media_pixfmt *
imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);
/* Find and enumerate media bus codes */
const struct imx_media_pixfmt *
imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
bool allow_bayer);
int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
bool allow_bayer);
/* Find and enumerate IPU internal media bus codes */
const struct imx_media_pixfmt *
imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);
The tables have been split into separate tables for YUV and RGB formats
to support the implementation of the above.
The subdev's .enum_mbus_code() and .set_fmt() operations have
been rewritten using the above APIs.
drivers/staging/media/imx/imx-ic-prp.c | 77 ++++--
drivers/staging/media/imx/imx-ic-prpencvf.c | 57 ++--
drivers/staging/media/imx/imx-media-capture.c | 85 ++++--
drivers/staging/media/imx/imx-media-csi.c | 108 +++++---
drivers/staging/media/imx/imx-media-utils.c | 371 +++++++++++++++++++-------
drivers/staging/media/imx/imx-media-vdic.c | 69 ++---
drivers/staging/media/imx/imx-media.h | 27 +-
7 files changed, 504 insertions(+), 290 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 1832915..83cd2b4 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -89,16 +89,6 @@ static void prp_stop(struct prp_priv *priv)
{
}
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= PRP_NUM_PADS)
- return -EINVAL;
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
static struct v4l2_mbus_framefmt *
* V4L2 subdev operations.
*/
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+
+ switch (code->pad) {
+ case PRP_SINK_PAD:
+ ret = imx_media_enum_ipu_format(&code->code, code->index,
+ CS_SEL_ANY);
+ break;
+ case PRP_SRC_PAD_PRPENC:
+ case PRP_SRC_PAD_PRPVF:
+ if (code->index != 0) {
+ code->code = infmt->code;
struct v4l2_subdev_format *sdformat)
@@ -160,23 +182,28 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
goto out;
}
- cc = imx_media_find_ipu_format(0, sdformat->format.code, true);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
- sdformat->format.code = cc->codes[0];
- }
-
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
- /* Output pads mirror input pad */
- if (sdformat->pad == PRP_SRC_PAD_PRPENC ||
- sdformat->pad == PRP_SRC_PAD_PRPVF) {
+ switch (sdformat->pad) {
+ case PRP_SINK_PAD:
+ CS_SEL_ANY);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+ cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ case PRP_SRC_PAD_PRPVF:
+ /* Output pads mirror input pad */
sdformat->format = *infmt;
+ break;
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
/* set a default mbus format */
- imx_media_enum_ipu_format(NULL, &code, 0, true);
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
&priv->cc[i]);
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
index bd89e9b..b42103c 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -702,20 +702,6 @@ static void prp_stop(struct prp_priv *priv)
prp_put_ipu_resources(priv);
}
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= PRPENCVF_NUM_PADS)
- return -EINVAL;
-
- if (code->pad == PRPENCVF_SRC_PAD)
- return imx_media_enum_format(NULL, &code->code, code->index,
- true, false);
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
static struct v4l2_mbus_framefmt *
* V4L2 subdev operations.
*/
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(&code->code, code->index, CS_SEL_ANY);
+}
+
struct v4l2_subdev_format *sdformat)
@@ -777,18 +773,17 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
goto out;
}
+ cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+ cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0, true, false);
- cc = imx_media_find_format(0, code, true, false);
- sdformat->format.code = cc->codes[0];
- }
-
} else {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- true);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
- sdformat->format.code = cc->codes[0];
- }
-
v4l_bound_align_image(&sdformat->format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
&sdformat->format.height,
@@ -1098,15 +1085,11 @@ static int prp_registered(struct v4l2_subdev *sd)
- if (i == PRPENCVF_SINK_PAD) {
- priv->pad[i].flags = MEDIA_PAD_FL_SINK;
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- } else {
- priv->pad[i].flags = MEDIA_PAD_FL_SOURCE;
- code = 0;
- }
+ priv->pad[i].flags = (i == PRPENCVF_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
/* set a default mbus format */
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
&priv->cc[i]);
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index 704ed85..a757e05 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -85,12 +85,39 @@ static int vidioc_querycap(struct file *file, void *fh,
+ struct v4l2_subdev_format fmt_src;
u32 fourcc;
int ret;
- ret = imx_media_enum_format(&fourcc, NULL, f->index, true, true);
- if (ret)
return ret;
+ }
+
+ cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+ if (!cc_src)
+ cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+ CS_SEL_ANY, true);
+ if (!cc_src)
+ return -EINVAL;
+
+ if (cc_src->bayer) {
+ if (f->index != 0)
+ return -EINVAL;
+ fourcc = cc_src->fourcc;
+ } else {
+ u32 cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+
+ ret = imx_media_enum_format(&fourcc, f->index, cs_sel);
@@ -112,40 +139,40 @@ static int capture_try_fmt_vid_cap(struct file *file, void *fh,
{
- const struct imx_media_pixfmt *cc, *src_cc;
- u32 fourcc;
+ const struct imx_media_pixfmt *cc, *cc_src;
int ret;
- fourcc = f->fmt.pix.pixelformat;
- cc = imx_media_find_format(fourcc, 0, true, true);
- if (!cc) {
- imx_media_enum_format(&fourcc, NULL, 0, true, true);
- cc = imx_media_find_format(fourcc, 0, true, true);
- }
-
- /*
- * user frame dimensions are the same as src_sd's pad.
- */
fmt_src.pad = priv->src_sd_pad;
fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
return ret;
- /*
- * but we can allow planar pixel formats if the src_sd's
- * pad configured a YUV format
- */
- src_cc = imx_media_find_format(0, fmt_src.format.code, true, false);
- if (src_cc->cs == IPUV3_COLORSPACE_YUV &&
- cc->cs == IPUV3_COLORSPACE_YUV) {
- imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
- &fmt_src.format, cc);
+ cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+ if (!cc_src)
+ cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+ CS_SEL_ANY, true);
+ if (!cc_src)
+ return -EINVAL;
+
+ if (cc_src->bayer) {
+ cc = cc_src;
} else {
- imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
- &fmt_src.format, src_cc);
+ u32 fourcc, cs_sel;
+
+ cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+ if (!cc) {
+ imx_media_enum_format(&fourcc, 0, cs_sel);
+ cc = imx_media_find_format(fourcc, cs_sel);
+ }
}
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix, &fmt_src.format, cc);
+
return 0;
}
@@ -165,8 +192,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void *fh,
return ret;
priv->vdev.fmt.fmt.pix = f->fmt.pix;
- priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat, 0,
- true, true);
return 0;
}
@@ -574,8 +601,8 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt.fmt.pix,
&fmt_src.format, NULL);
- vdev->cc = imx_media_find_format(0, fmt_src.format.code,
- true, false);
+ vdev->cc = imx_media_find_format(vdev->fmt.fmt.pix.pixelformat,
+ CS_SEL_ANY);
v4l2_info(sd, "Registered %s as /dev/%s\n", vfd->name,
video_device_node_name(vfd));
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index b556fa4..a726744 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -917,15 +917,44 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_mbus_code_enum *code)
{
- if (code->pad >= CSI_NUM_PADS)
- return -EINVAL;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
- if (code->pad == CSI_SRC_PAD_DIRECT)
- return imx_media_enum_ipu_format(NULL, &code->code,
- code->index, true);
+ mutex_lock(&priv->lock);
+
+ infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, code->which);
+ incc = imx_media_find_mbus_format(infmt->code, CS_SEL_ANY, true);
- return imx_media_enum_format(NULL, &code->code, code->index,
- true, false);
+ switch (code->pad) {
+ case CSI_SINK_PAD:
+ ret = imx_media_enum_mbus_format(&code->code, code->index,
+ CS_SEL_ANY, true);
+ break;
+ if (code->index != 0) {
+ } else {
+ u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+ ret = imx_media_enum_ipu_format(&code->code,
+ code->index,
+ cs_sel);
goto out;
}
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
+ infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD,
+ sdformat->which);
+ incc = imx_media_find_mbus_format(infmt->code,
+ CS_SEL_ANY, true);
if (sdformat->format.width < priv->crop.width * 3 / 4)
else
sdformat->format.height = priv->crop.height;
- if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0,
- true, false);
- cc = imx_media_find_format(0, code,
- true, false);
- sdformat->format.code = cc->codes[0];
- }
-
- incc = priv->cc[CSI_SINK_PAD];
- if (cc->cs != incc->cs) {
- sdformat->format.code = infmt->code;
- cc = imx_media_find_format(
- 0, sdformat->format.code,
- true, false);
- }
-
- if (sdformat->format.field != V4L2_FIELD_NONE)
- sdformat->format.field = infmt->field;
+ if (incc->bayer) {
} else {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- true);
+ u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+
+ cc = imx_media_find_ipu_format(sdformat->format.code,
+ cs_sel);
if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
+ imx_media_enum_ipu_format(&code, 0, cs_sel);
+ cc = imx_media_find_ipu_format(code, cs_sel);
/*
}
break;
case CSI_SINK_PAD:
crop.top = 0;
crop.width = sdformat->format.width;
@@ -1053,11 +1071,13 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
if (ret)
goto out;
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
+ cc = imx_media_find_mbus_format(sdformat->format.code,
+ CS_SEL_ANY, true);
if (!cc) {
- imx_media_enum_format(NULL, &code, 0, true, false);
- cc = imx_media_find_format(0, code, true, false);
+ imx_media_enum_mbus_format(&code, 0,
+ CS_SEL_ANY, false);
+ cc = imx_media_find_mbus_format(code,
+ CS_SEL_ANY, false);
@@ -1227,8 +1247,8 @@ static int csi_registered(struct v4l2_subdev *sd)
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
code = 0;
- if (i == CSI_SRC_PAD_DIRECT)
- imx_media_enum_ipu_format(NULL, &code, 0, true);
+ if (i != CSI_SINK_PAD)
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
/* set a default mbus format */
index 24e3795..a648fd5 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -12,14 +12,13 @@
#include "imx-media.h"
/*
- * List of pixel formats for the subdevs. This must be a super-set of
- * the formats supported by the ipu image converter.
+ * List of supported pixel formats for the subdevs.
*
- * The non-mbus formats (planar and BGR) must all fall at the end of
- * this table, otherwise enum_fmt() at media pads will stop before
- * seeing all the supported mbus formats.
+ * In all of these tables, the non-mbus formats (with no
+ * mbus codes) must all fall at the end of the table.
*/
-static const struct imx_media_pixfmt imx_media_formats[] = {
+
+static const struct imx_media_pixfmt yuv_formats[] = {
{
.fourcc = V4L2_PIX_FMT_UYVY,
.codes = {
@@ -36,13 +35,45 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
},
.cs = IPUV3_COLORSPACE_YUV,
.bpp = 16,
+ },
+ /***
+ * non-mbus YUV formats start here. NOTE! when adding non-mbus
+ * formats, NUM_NON_MBUS_YUV_FORMATS must be updated below.
+ ***/
+ {
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 12,
+ .planar = true,
}, {
- .fourcc = V4L2_PIX_FMT_YUV32,
- .codes = {MEDIA_BUS_FMT_AYUV8_1X32},
+ .fourcc = V4L2_PIX_FMT_YVU420,
.cs = IPUV3_COLORSPACE_YUV,
- .bpp = 32,
- .ipufmt = true,
+ .bpp = 12,
+ .planar = true,
}, {
+ .fourcc = V4L2_PIX_FMT_YUV422P,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 16,
+ .planar = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 12,
+ .planar = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV16,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 16,
+ .planar = true,
+ },
+};
+
+#define NUM_NON_MBUS_YUV_FORMATS 5
+#define NUM_YUV_FORMATS ARRAY_SIZE(yuv_formats)
+#define NUM_MBUS_YUV_FORMATS (NUM_YUV_FORMATS - NUM_NON_MBUS_YUV_FORMATS)
+
+static const struct imx_media_pixfmt rgb_formats[] = {
+ {
.fourcc = V4L2_PIX_FMT_RGB565,
.codes = {MEDIA_BUS_FMT_RGB565_2X8_LE},
.cs = IPUV3_COLORSPACE_RGB,
@@ -61,7 +92,9 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
+ },
+ /*** raw bayer formats start here ***/
+ {
.fourcc = V4L2_PIX_FMT_SBGGR8,
.codes = {MEDIA_BUS_FMT_SBGGR8_1X8},
.cs = IPUV3_COLORSPACE_RGB,
@@ -130,7 +163,10 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.bpp = 16,
.bayer = true,
},
- /*** non-mbus formats start here ***/
+ /***
+ * non-mbus RGB formats start here. NOTE! when adding non-mbus
+ * formats, NUM_NON_MBUS_RGB_FORMATS must be updated below.
+ ***/
{
.fourcc = V4L2_PIX_FMT_BGR24,
.cs = IPUV3_COLORSPACE_RGB,
@@ -139,135 +175,256 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.fourcc = V4L2_PIX_FMT_BGR32,
- .fourcc = V4L2_PIX_FMT_YUV420,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_YVU420,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_YUV422P,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 16,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_NV12,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_NV16,
+ },
+};
+
+#define NUM_NON_MBUS_RGB_FORMATS 2
+#define NUM_RGB_FORMATS ARRAY_SIZE(rgb_formats)
+#define NUM_MBUS_RGB_FORMATS (NUM_RGB_FORMATS - NUM_NON_MBUS_RGB_FORMATS)
+
+static const struct imx_media_pixfmt ipu_yuv_formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_YUV32,
+ .codes = {MEDIA_BUS_FMT_AYUV8_1X32},
.cs = IPUV3_COLORSPACE_YUV,
- .bpp = 16,
- .planar = true,
+ .bpp = 32,
+ .ipufmt = true,
},
};
-static const u32 imx_media_ipu_internal_codes[] = {
- MEDIA_BUS_FMT_AYUV8_1X32, MEDIA_BUS_FMT_ARGB8888_1X32,
+#define NUM_IPU_YUV_FORMATS ARRAY_SIZE(ipu_yuv_formats)
+
+static const struct imx_media_pixfmt ipu_rgb_formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_RGB32,
+ .codes = {MEDIA_BUS_FMT_ARGB8888_1X32},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 32,
+ .ipufmt = true,
+ },
};
+#define NUM_IPU_RGB_FORMATS ARRAY_SIZE(ipu_rgb_formats)
+
static inline u32 pixfmt_to_colorspace(const struct imx_media_pixfmt *fmt)
{
return (fmt->cs == IPUV3_COLORSPACE_RGB) ?
V4L2_COLORSPACE_SRGB : V4L2_COLORSPACE_SMPTE170M;
}
-static const struct imx_media_pixfmt *find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar,
- bool ipu_fmt_only)
+static const struct imx_media_pixfmt *find_format(u32 fourcc,
+ u32 code,
+ enum codespace_sel cs_sel,
+ bool allow_non_mbus,
+ bool allow_bayer)
{
- const struct imx_media_pixfmt *fmt, *ret = NULL;
+ const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+ u32 array_size;
int i, j;
- for (i = 0; i < ARRAY_SIZE(imx_media_formats); i++) {
- fmt = &imx_media_formats[i];
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ array_size = NUM_YUV_FORMATS;
+ array = yuv_formats;
+ break;
+ case CS_SEL_RGB:
+ array_size = NUM_RGB_FORMATS;
+ array = rgb_formats;
+ break;
+ case CS_SEL_ANY:
+ array_size = NUM_YUV_FORMATS + NUM_RGB_FORMATS;
+ array = yuv_formats;
+ break;
+ default:
+ return NULL;
+ }
+
+ for (i = 0; i < array_size; i++) {
+ if (cs_sel == CS_SEL_ANY && i >= NUM_YUV_FORMATS)
+ fmt = &rgb_formats[i - NUM_YUV_FORMATS];
+ else
+ fmt = &array[i];
- if (ipu_fmt_only && !fmt->ipufmt)
+ if ((!allow_non_mbus && fmt->codes[0] == 0) ||
+ (!allow_bayer && fmt->bayer))
continue;
- if (fourcc && fmt->fourcc == fourcc &&
- (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb) &&
- (!fmt->planar || allow_planar)) {
+ if (fourcc && fmt->fourcc == fourcc) {
ret = fmt;
goto out;
}
for (j = 0; code && fmt->codes[j]; j++) {
- if (fmt->codes[j] == code && !fmt->planar &&
- (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb)) {
+ if (code == fmt->codes[j]) {
ret = fmt;
goto out;
}
}
}
+
out:
return ret;
}
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar)
+static int enum_format(u32 *fourcc, u32 *code, u32 index,
+ enum codespace_sel cs_sel,
+ bool allow_non_mbus,
+ bool allow_bayer)
+{
+ const struct imx_media_pixfmt *fmt;
+ u32 mbus_yuv_sz = NUM_MBUS_YUV_FORMATS;
+ u32 mbus_rgb_sz = NUM_MBUS_RGB_FORMATS;
+ u32 yuv_sz = NUM_YUV_FORMATS;
+ u32 rgb_sz = NUM_RGB_FORMATS;
+
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ if (index >= yuv_sz ||
+ (!allow_non_mbus && index >= mbus_yuv_sz))
+ return -EINVAL;
+ fmt = &yuv_formats[index];
+ break;
+ case CS_SEL_RGB:
+ if (index >= rgb_sz ||
+ (!allow_non_mbus && index >= mbus_rgb_sz))
+ return -EINVAL;
+ fmt = &rgb_formats[index];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ break;
+ case CS_SEL_ANY:
+ if (!allow_non_mbus) {
+ if (index >= mbus_yuv_sz) {
+ index -= mbus_yuv_sz;
+ if (index >= mbus_rgb_sz)
+ return -EINVAL;
+ fmt = &rgb_formats[index];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ } else {
+ fmt = &yuv_formats[index];
+ }
+ } else {
+ if (index >= yuv_sz + rgb_sz)
+ return -EINVAL;
+ if (index >= yuv_sz) {
+ fmt = &rgb_formats[index - yuv_sz];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ } else {
+ fmt = &yuv_formats[index];
+ }
+ *fourcc = fmt->fourcc;
+ if (code)
+ *code = fmt->codes[0];
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel)
{
- return find_format(fourcc, code, allow_rgb, allow_planar, false);
+ return find_format(fourcc, 0, cs_sel, true, false);
}
EXPORT_SYMBOL_GPL(imx_media_find_format);
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc,
- u32 code,
- bool allow_rgb)
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel)
{
- return find_format(fourcc, code, allow_rgb, false, true);
+ return enum_format(fourcc, NULL, index, cs_sel, true, false);
}
-EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
+EXPORT_SYMBOL_GPL(imx_media_enum_format);
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb, bool allow_planar)
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+ bool allow_bayer)
{
- const struct imx_media_pixfmt *fmt;
+ return find_format(0, code, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_find_mbus_format);
- if (index >= ARRAY_SIZE(imx_media_formats))
- return -EINVAL;
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+ bool allow_bayer)
+{
+ return enum_format(NULL, code, index, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_enum_mbus_format);
- fmt = &imx_media_formats[index];
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel)
+{
+ const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+ u32 array_size;
+ int i, j;
- if ((fmt->cs == IPUV3_COLORSPACE_RGB && !allow_rgb) ||
- (fmt->planar && !allow_planar))
- return -EINVAL;
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ array_size = NUM_IPU_YUV_FORMATS;
+ array = ipu_yuv_formats;
+ break;
+ case CS_SEL_RGB:
+ array_size = NUM_IPU_RGB_FORMATS;
+ array = ipu_rgb_formats;
+ break;
+ case CS_SEL_ANY:
+ array_size = NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS;
+ array = ipu_yuv_formats;
+ break;
+ default:
+ return NULL;
+ }
- if (code)
- *code = fmt->codes[0];
- if (fourcc)
- *fourcc = fmt->fourcc;
+ for (i = 0; i < array_size; i++) {
+ if (cs_sel == CS_SEL_ANY && i >= NUM_IPU_YUV_FORMATS)
+ fmt = &ipu_rgb_formats[i - NUM_IPU_YUV_FORMATS];
+ else
+ fmt = &array[i];
- return 0;
+ for (j = 0; code && fmt->codes[j]; j++) {
+ if (code == fmt->codes[j]) {
+ ret = fmt;
+ return ret;
}
-EXPORT_SYMBOL_GPL(imx_media_enum_format);
+EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb)
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel)
{
- const struct imx_media_pixfmt *fmt;
- u32 lcode;
-
- if (index >= ARRAY_SIZE(imx_media_ipu_internal_codes))
- return -EINVAL;
-
- lcode = imx_media_ipu_internal_codes[index];
-
- fmt = find_format(0, lcode, allow_rgb, false, true);
- if (!fmt)
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ if (index >= NUM_IPU_YUV_FORMATS)
+ return -EINVAL;
+ *code = ipu_yuv_formats[index].codes[0];
+ break;
+ case CS_SEL_RGB:
+ if (index >= NUM_IPU_RGB_FORMATS)
+ return -EINVAL;
+ *code = ipu_rgb_formats[index].codes[0];
+ break;
+ case CS_SEL_ANY:
+ if (index >= NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS)
+ return -EINVAL;
+ if (index >= NUM_IPU_YUV_FORMATS) {
+ index -= NUM_IPU_YUV_FORMATS;
+ *code = ipu_rgb_formats[index].codes[0];
+ } else {
+ *code = ipu_yuv_formats[index].codes[0];
-
- if (code)
- *code = fmt->codes[0];
- if (fourcc)
- *fourcc = fmt->fourcc;
+ }
return 0;
}
@@ -283,10 +440,14 @@ int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
mbus->height = height;
mbus->field = field;
if (code == 0)
- imx_media_enum_format(NULL, &code, 0, true, false);
- lcc = imx_media_find_format(0, code, true, false);
- if (!lcc)
- return -EINVAL;
+ imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+ lcc = imx_media_find_mbus_format(code, CS_SEL_ANY, false);
+ if (!lcc) {
+ lcc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ if (!lcc)
mbus->colorspace = pixfmt_to_colorspace(lcc);
@@ -304,11 +465,25 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
u32 stride;
if (!cc) {
- cc = imx_media_find_format(0, mbus->code, true, false);
+ cc = imx_media_find_ipu_format(mbus->code, CS_SEL_ANY);
+ if (!cc)
+ cc = imx_media_find_mbus_format(mbus->code, CS_SEL_ANY,
+ true);
if (!cc)
return -EINVAL;
}
+ /*
+ * TODO: the IPU currently does not support the AYUV32 format,
+ * so until it does convert to a supported YUV format.
+ */
+ if (cc->ipufmt && cc->cs == IPUV3_COLORSPACE_YUV) {
+ u32 code;
+
+ imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+ cc = imx_media_find_mbus_format(code, CS_SEL_YUV, false);
+ }
+
stride = cc->planar ? mbus->width : (mbus->width * cc->bpp) >> 3;
pix->width = mbus->width;
@@ -349,7 +524,7 @@ int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
{
const struct imx_media_pixfmt *fmt;
- fmt = imx_media_find_format(image->pix.pixelformat, 0, true, false);
+ fmt = imx_media_find_format(image->pix.pixelformat, CS_SEL_ANY);
if (!fmt)
return -EINVAL;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 0e7c24c..070f9da 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -517,20 +517,6 @@ static int vdic_s_stream(struct v4l2_subdev *sd, int enable)
return ret;
}
-static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= VDIC_NUM_PADS)
- return -EINVAL;
-
- if (code->pad == VDIC_SINK_PAD_IDMAC)
- return imx_media_enum_format(NULL, &code->code, code->index,
- false, false);
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, false);
-}
-
static struct v4l2_mbus_framefmt *
return &priv->format_mbus[pad];
}
+static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(&code->code, code->index, CS_SEL_YUV);
+}
+
struct v4l2_subdev_format *sdformat)
@@ -586,49 +582,28 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
goto out;
}
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W_VDIC,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
+ cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
+ cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
case VDIC_SRC_PAD_DIRECT:
-
- cc = imx_media_find_ipu_format(0, sdformat->format.code, false);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, false);
- cc = imx_media_find_ipu_format(0, code, false);
- sdformat->format.code = cc->codes[0];
- }
-
break;
- case VDIC_SINK_PAD_IDMAC:
case VDIC_SINK_PAD_DIRECT:
- if (sdformat->pad == VDIC_SINK_PAD_DIRECT) {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- false);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0,
- false);
- cc = imx_media_find_ipu_format(0, code, false);
- sdformat->format.code = cc->codes[0];
- }
- } else {
- cc = imx_media_find_format(0, sdformat->format.code,
- false, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0,
- false, false);
- cc = imx_media_find_format(0, code,
- false, false);
- sdformat->format.code = cc->codes[0];
- }
- }
+ case VDIC_SINK_PAD_IDMAC:
+ v4l_bound_align_image(&sdformat->format.width,
+ MIN_W, MAX_W_VDIC, W_ALIGN,
+ &sdformat->format.height,
@@ -814,7 +789,7 @@ static int vdic_registered(struct v4l2_subdev *sd)
code = 0;
if (i != VDIC_SINK_PAD_IDMAC)
- imx_media_enum_ipu_format(NULL, &code, 0, true);
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
/* set a default mbus format */
index a67ee14..2342422 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -171,16 +171,23 @@ struct imx_media_dev {
struct v4l2_async_notifier subdev_notifier;
};
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar);
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc, u32 code,
- bool allow_rgb);
-
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb, bool allow_planar);
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb);
+enum codespace_sel {
+ CS_SEL_YUV = 0,
+ CS_SEL_RGB,
+ CS_SEL_ANY,
+};
+
+const struct imx_media_pixfmt *
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+ bool allow_bayer);
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+ bool allow_bayer);
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);
int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
u32 width, u32 height, u32 code, u32 field,
--
2.7.4
was broken, and has been completely redone.
The negotiation of media bus codes should be as follows:
CSI:
sink pad direct src pad IDMAC src pad
-------- ---------------- -------------
RGB (any) IPU RGB IPU RGB
YUV (any) IPU YUV IPU YUV
Bayer N/A must be same bayer code as sink
VDIC:
direct sink pad IDMAC sink pad direct src pad
--------------- -------------- --------------
IPU YUV only IPU YUV only IPU YUV only
PRP:
direct sink pad direct src pads
--------------- ---------------
IPU (any) same as sink code
PRP ENC/VF:
direct sink pad IDMAC src pads
--------------- --------------
IPU (any) IPU RGB or IPU YUV
Given the above, a new internal API is created:
enum codespace_sel {
CS_SEL_YUV = 0, /* find or enumerate only YUV codes */
CS_SEL_RGB, /* find or enumerate only RGB codes */
CS_SEL_ANY, /* find or enumerate both YUV and RGB codes */
};
/* Find and enumerate fourcc pixel formats */
const struct imx_media_pixfmt *
imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);
/* Find and enumerate media bus codes */
const struct imx_media_pixfmt *
imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
bool allow_bayer);
int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
bool allow_bayer);
/* Find and enumerate IPU internal media bus codes */
const struct imx_media_pixfmt *
imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);
The tables have been split into separate tables for YUV and RGB formats
to support the implementation of the above.
The subdev's .enum_mbus_code() and .set_fmt() operations have
been rewritten using the above APIs.
drivers/staging/media/imx/imx-ic-prp.c | 77 ++++--
drivers/staging/media/imx/imx-ic-prpencvf.c | 57 ++--
drivers/staging/media/imx/imx-media-capture.c | 85 ++++--
drivers/staging/media/imx/imx-media-csi.c | 108 +++++---
drivers/staging/media/imx/imx-media-utils.c | 371 +++++++++++++++++++-------
drivers/staging/media/imx/imx-media-vdic.c | 69 ++---
drivers/staging/media/imx/imx-media.h | 27 +-
7 files changed, 504 insertions(+), 290 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 1832915..83cd2b4 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -89,16 +89,6 @@ static void prp_stop(struct prp_priv *priv)
{
}
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= PRP_NUM_PADS)
- return -EINVAL;
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
static struct v4l2_mbus_framefmt *
__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -115,6 +105,38 @@ __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
* V4L2 subdev operations.
*/
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct prp_priv *priv = sd_to_priv(sd);
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
+
+ mutex_lock(&priv->lock);
+ int ret = 0;
+
+
+ switch (code->pad) {
+ case PRP_SINK_PAD:
+ ret = imx_media_enum_ipu_format(&code->code, code->index,
+ CS_SEL_ANY);
+ break;
+ case PRP_SRC_PAD_PRPENC:
+ case PRP_SRC_PAD_PRPVF:
+ if (code->index != 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+ infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD, code->which);
+ goto out;
+ }
+ code->code = infmt->code;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ default:
+ ret = -EINVAL;
+ }
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
+ mutex_unlock(&priv->lock);
+ return ret;
+}
+
static int prp_get_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat)
@@ -160,23 +182,28 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
goto out;
}
- cc = imx_media_find_ipu_format(0, sdformat->format.code, true);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
- sdformat->format.code = cc->codes[0];
- }
-
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
- /* Output pads mirror input pad */
- if (sdformat->pad == PRP_SRC_PAD_PRPENC ||
- sdformat->pad == PRP_SRC_PAD_PRPVF) {
+ switch (sdformat->pad) {
+ case PRP_SINK_PAD:
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+ cc = imx_media_find_ipu_format(sdformat->format.code,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+ CS_SEL_ANY);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+ cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ sdformat->format.code = cc->codes[0];
+ }
+ break;
+ case PRP_SRC_PAD_PRPENC:
+ }
+ break;
+ case PRP_SRC_PAD_PRPVF:
+ /* Output pads mirror input pad */
infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD,
sdformat->which);
+ cc = imx_media_find_ipu_format(infmt->code, CS_SEL_ANY);
sdformat->which);
sdformat->format = *infmt;
+ break;
}
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
@@ -402,7 +429,7 @@ static int prp_registered(struct v4l2_subdev *sd)
if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY) {
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
/* set a default mbus format */
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
640, 480, code, V4L2_FIELD_NONE,
&priv->cc[i]);
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
index bd89e9b..b42103c 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -702,20 +702,6 @@ static void prp_stop(struct prp_priv *priv)
prp_put_ipu_resources(priv);
}
-static int prp_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= PRPENCVF_NUM_PADS)
- return -EINVAL;
-
- if (code->pad == PRPENCVF_SRC_PAD)
- return imx_media_enum_format(NULL, &code->code, code->index,
- true, false);
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, true);
-}
-
static struct v4l2_mbus_framefmt *
__prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -732,6 +718,16 @@ __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_pad_config *cfg,
* V4L2 subdev operations.
*/
+static int prp_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= PRPENCVF_NUM_PADS)
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(&code->code, code->index, CS_SEL_ANY);
+}
+
static int prp_get_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat)
@@ -777,18 +773,17 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
goto out;
}
+ cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_ANY);
+ cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ sdformat->format.code = cc->codes[0];
+ }
+
+ }
+
if (sdformat->pad == PRPENCVF_SRC_PAD) {
infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
sdformat->which);
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0, true, false);
- cc = imx_media_find_format(0, code, true, false);
- sdformat->format.code = cc->codes[0];
- }
-
if (sdformat->format.field != V4L2_FIELD_NONE)
sdformat->format.field = infmt->field;
@@ -808,14 +803,6 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
infmt->height / 4, MAX_H_SRC,
H_ALIGN_SRC, S_ALIGN);
} else {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- true);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
- sdformat->format.code = cc->codes[0];
- }
-
v4l_bound_align_image(&sdformat->format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
&sdformat->format.height,
@@ -1098,15 +1085,11 @@ static int prp_registered(struct v4l2_subdev *sd)
priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
- if (i == PRPENCVF_SINK_PAD) {
- priv->pad[i].flags = MEDIA_PAD_FL_SINK;
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- } else {
- priv->pad[i].flags = MEDIA_PAD_FL_SOURCE;
- code = 0;
- }
+ priv->pad[i].flags = (i == PRPENCVF_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
/* set a default mbus format */
ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
640, 480, code, V4L2_FIELD_NONE,
&priv->cc[i]);
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index 704ed85..a757e05 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -85,12 +85,39 @@ static int vidioc_querycap(struct file *file, void *fh,
static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
struct v4l2_fmtdesc *f)
{
+ struct capture_priv *priv = video_drvdata(file);
+ const struct imx_media_pixfmt *cc_src;
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_format fmt_src;
u32 fourcc;
int ret;
- ret = imx_media_enum_format(&fourcc, NULL, f->index, true, true);
- if (ret)
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
+ if (ret) {
+ v4l2_err(priv->src_sd, "failed to get src_sd format\n");
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
+ if (ret) {
return ret;
+ }
+
+ cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+ if (!cc_src)
+ cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+ CS_SEL_ANY, true);
+ if (!cc_src)
+ return -EINVAL;
+
+ if (cc_src->bayer) {
+ if (f->index != 0)
+ return -EINVAL;
+ fourcc = cc_src->fourcc;
+ } else {
+ u32 cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+
+ ret = imx_media_enum_format(&fourcc, f->index, cs_sel);
+ if (ret)
+ return ret;
+ }
f->pixelformat = fourcc;
+ return ret;
+ }
@@ -112,40 +139,40 @@ static int capture_try_fmt_vid_cap(struct file *file, void *fh,
{
struct capture_priv *priv = video_drvdata(file);
struct v4l2_subdev_format fmt_src;
- const struct imx_media_pixfmt *cc, *src_cc;
- u32 fourcc;
+ const struct imx_media_pixfmt *cc, *cc_src;
int ret;
- fourcc = f->fmt.pix.pixelformat;
- cc = imx_media_find_format(fourcc, 0, true, true);
- if (!cc) {
- imx_media_enum_format(&fourcc, NULL, 0, true, true);
- cc = imx_media_find_format(fourcc, 0, true, true);
- }
-
- /*
- * user frame dimensions are the same as src_sd's pad.
- */
fmt_src.pad = priv->src_sd_pad;
fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
if (ret)
return ret;
- /*
- * but we can allow planar pixel formats if the src_sd's
- * pad configured a YUV format
- */
- src_cc = imx_media_find_format(0, fmt_src.format.code, true, false);
- if (src_cc->cs == IPUV3_COLORSPACE_YUV &&
- cc->cs == IPUV3_COLORSPACE_YUV) {
- imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
- &fmt_src.format, cc);
+ cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+ if (!cc_src)
+ cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+ CS_SEL_ANY, true);
+ if (!cc_src)
+ return -EINVAL;
+
+ if (cc_src->bayer) {
+ cc = cc_src;
} else {
- imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix,
- &fmt_src.format, src_cc);
+ u32 fourcc, cs_sel;
+
+ cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+ fourcc = f->fmt.pix.pixelformat;
+
+ cc = imx_media_find_format(fourcc, cs_sel);
+
+ if (!cc) {
+ imx_media_enum_format(&fourcc, 0, cs_sel);
+ cc = imx_media_find_format(fourcc, cs_sel);
+ }
}
+ imx_media_mbus_fmt_to_pix_fmt(&f->fmt.pix, &fmt_src.format, cc);
+
return 0;
}
@@ -165,8 +192,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void *fh,
return ret;
priv->vdev.fmt.fmt.pix = f->fmt.pix;
- true, true);
+ priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat,
+ CS_SEL_ANY);
return 0;
}
@@ -574,8 +601,8 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt.fmt.pix,
&fmt_src.format, NULL);
- vdev->cc = imx_media_find_format(0, fmt_src.format.code,
- true, false);
+ vdev->cc = imx_media_find_format(vdev->fmt.fmt.pix.pixelformat,
+ CS_SEL_ANY);
v4l2_info(sd, "Registered %s as /dev/%s\n", vfd->name,
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index b556fa4..a726744 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -917,15 +917,44 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_mbus_code_enum *code)
{
- if (code->pad >= CSI_NUM_PADS)
- return -EINVAL;
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ const struct imx_media_pixfmt *incc;
+ struct v4l2_mbus_framefmt *infmt;
+ int ret = 0;
- if (code->pad == CSI_SRC_PAD_DIRECT)
- return imx_media_enum_ipu_format(NULL, &code->code,
- code->index, true);
+ mutex_lock(&priv->lock);
+
+ infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, code->which);
+ incc = imx_media_find_mbus_format(infmt->code, CS_SEL_ANY, true);
- return imx_media_enum_format(NULL, &code->code, code->index,
- true, false);
+ switch (code->pad) {
+ case CSI_SINK_PAD:
+ ret = imx_media_enum_mbus_format(&code->code, code->index,
+ CS_SEL_ANY, true);
+ break;
+ case CSI_SRC_PAD_DIRECT:
+ case CSI_SRC_PAD_IDMAC:
+ if (incc->bayer) {
+ case CSI_SRC_PAD_IDMAC:
+ if (code->index != 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+ code->code = infmt->code;
+ goto out;
+ }
+ } else {
+ u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+ ret = imx_media_enum_ipu_format(&code->code,
+ code->index,
+ cs_sel);
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
}
+out:
+ mutex_unlock(&priv->lock);
+ return ret;
}
static int csi_get_fmt(struct v4l2_subdev *sd,
@@ -981,14 +1010,13 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
goto out;
}
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H, H_ALIGN, S_ALIGN);
-
switch (sdformat->pad) {
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
- infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
case CSI_SRC_PAD_DIRECT:
case CSI_SRC_PAD_IDMAC:
+ infmt = __csi_get_fmt(priv, cfg, CSI_SINK_PAD,
+ sdformat->which);
+ incc = imx_media_find_mbus_format(infmt->code,
+ CS_SEL_ANY, true);
if (sdformat->format.width < priv->crop.width * 3 / 4)
sdformat->format.width = priv->crop.width / 2;
@@ -1000,38 +1028,25 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
else
sdformat->format.height = priv->crop.height;
- if (sdformat->pad == CSI_SRC_PAD_IDMAC) {
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0,
- true, false);
- cc = imx_media_find_format(0, code,
- true, false);
- sdformat->format.code = cc->codes[0];
- }
-
- incc = priv->cc[CSI_SINK_PAD];
- if (cc->cs != incc->cs) {
- sdformat->format.code = infmt->code;
- cc = imx_media_find_format(
- 0, sdformat->format.code,
- true, false);
- }
-
- if (sdformat->format.field != V4L2_FIELD_NONE)
- sdformat->format.field = infmt->field;
+ if (incc->bayer) {
+ sdformat->format.code = infmt->code;
+ cc = incc;
} else {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- true);
+ u32 cs_sel = (incc->cs == IPUV3_COLORSPACE_YUV) ?
+ CS_SEL_YUV : CS_SEL_RGB;
+
+ cc = imx_media_find_ipu_format(sdformat->format.code,
+ cs_sel);
if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, true);
- cc = imx_media_find_ipu_format(0, code, true);
+ imx_media_enum_ipu_format(&code, 0, cs_sel);
+ cc = imx_media_find_ipu_format(code, cs_sel);
sdformat->format.code = cc->codes[0];
}
+ }
}
+ }
+ if (sdformat->pad == CSI_SRC_PAD_DIRECT ||
+ sdformat->format.field != V4L2_FIELD_NONE)
sdformat->format.field = infmt->field;
- }
/*
* translate V4L2_FIELD_ALTERNATE to SEQ_TB or SEQ_BT
@@ -1044,6 +1059,9 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
}
break;
case CSI_SINK_PAD:
+ v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
crop.left = 0;
+ W_ALIGN, &sdformat->format.height,
+ MIN_H, MAX_H, H_ALIGN, S_ALIGN);
crop.top = 0;
crop.width = sdformat->format.width;
@@ -1053,11 +1071,13 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
if (ret)
goto out;
- cc = imx_media_find_format(0, sdformat->format.code,
- true, false);
+ cc = imx_media_find_mbus_format(sdformat->format.code,
+ CS_SEL_ANY, true);
if (!cc) {
- imx_media_enum_format(NULL, &code, 0, true, false);
- cc = imx_media_find_format(0, code, true, false);
+ imx_media_enum_mbus_format(&code, 0,
+ CS_SEL_ANY, false);
+ cc = imx_media_find_mbus_format(code,
+ CS_SEL_ANY, false);
sdformat->format.code = cc->codes[0];
}
break;
}
@@ -1227,8 +1247,8 @@ static int csi_registered(struct v4l2_subdev *sd)
MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
code = 0;
- if (i == CSI_SRC_PAD_DIRECT)
- imx_media_enum_ipu_format(NULL, &code, 0, true);
+ if (i != CSI_SINK_PAD)
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
/* set a default mbus format */
ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 24e3795..a648fd5 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -12,14 +12,13 @@
#include "imx-media.h"
/*
- * List of pixel formats for the subdevs. This must be a super-set of
- * the formats supported by the ipu image converter.
+ * List of supported pixel formats for the subdevs.
*
- * The non-mbus formats (planar and BGR) must all fall at the end of
- * this table, otherwise enum_fmt() at media pads will stop before
- * seeing all the supported mbus formats.
+ * In all of these tables, the non-mbus formats (with no
+ * mbus codes) must all fall at the end of the table.
*/
-static const struct imx_media_pixfmt imx_media_formats[] = {
+
+static const struct imx_media_pixfmt yuv_formats[] = {
{
.fourcc = V4L2_PIX_FMT_UYVY,
.codes = {
@@ -36,13 +35,45 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
},
.cs = IPUV3_COLORSPACE_YUV,
.bpp = 16,
+ },
+ /***
+ * non-mbus YUV formats start here. NOTE! when adding non-mbus
+ * formats, NUM_NON_MBUS_YUV_FORMATS must be updated below.
+ ***/
+ {
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 12,
+ .planar = true,
}, {
- .fourcc = V4L2_PIX_FMT_YUV32,
- .codes = {MEDIA_BUS_FMT_AYUV8_1X32},
+ .fourcc = V4L2_PIX_FMT_YVU420,
.cs = IPUV3_COLORSPACE_YUV,
- .bpp = 32,
- .ipufmt = true,
+ .bpp = 12,
+ .planar = true,
}, {
+ .fourcc = V4L2_PIX_FMT_YUV422P,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 16,
+ .planar = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 12,
+ .planar = true,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV16,
+ .cs = IPUV3_COLORSPACE_YUV,
+ .bpp = 16,
+ .planar = true,
+ },
+};
+
+#define NUM_NON_MBUS_YUV_FORMATS 5
+#define NUM_YUV_FORMATS ARRAY_SIZE(yuv_formats)
+#define NUM_MBUS_YUV_FORMATS (NUM_YUV_FORMATS - NUM_NON_MBUS_YUV_FORMATS)
+
+static const struct imx_media_pixfmt rgb_formats[] = {
+ {
.fourcc = V4L2_PIX_FMT_RGB565,
.codes = {MEDIA_BUS_FMT_RGB565_2X8_LE},
.cs = IPUV3_COLORSPACE_RGB,
@@ -61,7 +92,9 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.cs = IPUV3_COLORSPACE_RGB,
.bpp = 32,
.ipufmt = true,
- }, {
.bpp = 32,
.ipufmt = true,
+ },
+ /*** raw bayer formats start here ***/
+ {
.fourcc = V4L2_PIX_FMT_SBGGR8,
.codes = {MEDIA_BUS_FMT_SBGGR8_1X8},
.cs = IPUV3_COLORSPACE_RGB,
@@ -130,7 +163,10 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.bpp = 16,
.bayer = true,
},
- /*** non-mbus formats start here ***/
+ /***
+ * non-mbus RGB formats start here. NOTE! when adding non-mbus
+ * formats, NUM_NON_MBUS_RGB_FORMATS must be updated below.
+ ***/
{
.fourcc = V4L2_PIX_FMT_BGR24,
.cs = IPUV3_COLORSPACE_RGB,
@@ -139,135 +175,256 @@ static const struct imx_media_pixfmt imx_media_formats[] = {
.fourcc = V4L2_PIX_FMT_BGR32,
.cs = IPUV3_COLORSPACE_RGB,
.bpp = 32,
- }, {
.bpp = 32,
- .fourcc = V4L2_PIX_FMT_YUV420,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_YVU420,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_YUV422P,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 16,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_NV12,
- .cs = IPUV3_COLORSPACE_YUV,
- .bpp = 12,
- .planar = true,
- }, {
- .fourcc = V4L2_PIX_FMT_NV16,
+ },
+};
+
+#define NUM_NON_MBUS_RGB_FORMATS 2
+#define NUM_RGB_FORMATS ARRAY_SIZE(rgb_formats)
+#define NUM_MBUS_RGB_FORMATS (NUM_RGB_FORMATS - NUM_NON_MBUS_RGB_FORMATS)
+
+static const struct imx_media_pixfmt ipu_yuv_formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_YUV32,
+ .codes = {MEDIA_BUS_FMT_AYUV8_1X32},
.cs = IPUV3_COLORSPACE_YUV,
- .bpp = 16,
- .planar = true,
+ .bpp = 32,
+ .ipufmt = true,
},
};
-static const u32 imx_media_ipu_internal_codes[] = {
- MEDIA_BUS_FMT_AYUV8_1X32, MEDIA_BUS_FMT_ARGB8888_1X32,
+#define NUM_IPU_YUV_FORMATS ARRAY_SIZE(ipu_yuv_formats)
+
+static const struct imx_media_pixfmt ipu_rgb_formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_RGB32,
+ .codes = {MEDIA_BUS_FMT_ARGB8888_1X32},
+ .cs = IPUV3_COLORSPACE_RGB,
+ .bpp = 32,
+ .ipufmt = true,
+ },
};
+#define NUM_IPU_RGB_FORMATS ARRAY_SIZE(ipu_rgb_formats)
+
static inline u32 pixfmt_to_colorspace(const struct imx_media_pixfmt *fmt)
{
return (fmt->cs == IPUV3_COLORSPACE_RGB) ?
V4L2_COLORSPACE_SRGB : V4L2_COLORSPACE_SMPTE170M;
}
-static const struct imx_media_pixfmt *find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar,
- bool ipu_fmt_only)
+static const struct imx_media_pixfmt *find_format(u32 fourcc,
+ u32 code,
+ enum codespace_sel cs_sel,
+ bool allow_non_mbus,
+ bool allow_bayer)
{
- const struct imx_media_pixfmt *fmt, *ret = NULL;
+ const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+ u32 array_size;
int i, j;
- for (i = 0; i < ARRAY_SIZE(imx_media_formats); i++) {
- fmt = &imx_media_formats[i];
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ array_size = NUM_YUV_FORMATS;
+ array = yuv_formats;
+ break;
+ case CS_SEL_RGB:
+ array_size = NUM_RGB_FORMATS;
+ array = rgb_formats;
+ break;
+ case CS_SEL_ANY:
+ array_size = NUM_YUV_FORMATS + NUM_RGB_FORMATS;
+ array = yuv_formats;
+ break;
+ default:
+ return NULL;
+ }
+
+ for (i = 0; i < array_size; i++) {
+ if (cs_sel == CS_SEL_ANY && i >= NUM_YUV_FORMATS)
+ fmt = &rgb_formats[i - NUM_YUV_FORMATS];
+ else
+ fmt = &array[i];
- if (ipu_fmt_only && !fmt->ipufmt)
+ if ((!allow_non_mbus && fmt->codes[0] == 0) ||
+ (!allow_bayer && fmt->bayer))
continue;
- if (fourcc && fmt->fourcc == fourcc &&
- (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb) &&
- (!fmt->planar || allow_planar)) {
+ if (fourcc && fmt->fourcc == fourcc) {
ret = fmt;
goto out;
}
for (j = 0; code && fmt->codes[j]; j++) {
- if (fmt->codes[j] == code && !fmt->planar &&
- (fmt->cs != IPUV3_COLORSPACE_RGB || allow_rgb)) {
+ if (code == fmt->codes[j]) {
ret = fmt;
goto out;
}
}
}
+
out:
return ret;
}
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar)
+static int enum_format(u32 *fourcc, u32 *code, u32 index,
+ enum codespace_sel cs_sel,
+ bool allow_non_mbus,
+ bool allow_bayer)
+{
+ const struct imx_media_pixfmt *fmt;
+ u32 mbus_yuv_sz = NUM_MBUS_YUV_FORMATS;
+ u32 mbus_rgb_sz = NUM_MBUS_RGB_FORMATS;
+ u32 yuv_sz = NUM_YUV_FORMATS;
+ u32 rgb_sz = NUM_RGB_FORMATS;
+
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ if (index >= yuv_sz ||
+ (!allow_non_mbus && index >= mbus_yuv_sz))
+ return -EINVAL;
+ fmt = &yuv_formats[index];
+ break;
+ case CS_SEL_RGB:
+ if (index >= rgb_sz ||
+ (!allow_non_mbus && index >= mbus_rgb_sz))
+ return -EINVAL;
+ fmt = &rgb_formats[index];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ break;
+ case CS_SEL_ANY:
+ if (!allow_non_mbus) {
+ if (index >= mbus_yuv_sz) {
+ index -= mbus_yuv_sz;
+ if (index >= mbus_rgb_sz)
+ return -EINVAL;
+ fmt = &rgb_formats[index];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ } else {
+ fmt = &yuv_formats[index];
+ }
+ } else {
+ if (index >= yuv_sz + rgb_sz)
+ return -EINVAL;
+ if (index >= yuv_sz) {
+ fmt = &rgb_formats[index - yuv_sz];
+ if (!allow_bayer && fmt->bayer)
+ return -EINVAL;
+ } else {
+ fmt = &yuv_formats[index];
+ }
+ }
+ break;
+ default:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (fourcc)
+ }
+
+ *fourcc = fmt->fourcc;
+ if (code)
+ *code = fmt->codes[0];
+
+ return 0;
+}
+
+const struct imx_media_pixfmt *
+ return 0;
+}
+
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel)
{
- return find_format(fourcc, code, allow_rgb, allow_planar, false);
+ return find_format(fourcc, 0, cs_sel, true, false);
}
EXPORT_SYMBOL_GPL(imx_media_find_format);
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc,
- u32 code,
- bool allow_rgb)
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel)
{
- return find_format(fourcc, code, allow_rgb, false, true);
+ return enum_format(fourcc, NULL, index, cs_sel, true, false);
}
-EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
+EXPORT_SYMBOL_GPL(imx_media_enum_format);
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb, bool allow_planar)
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+ bool allow_bayer)
{
- const struct imx_media_pixfmt *fmt;
+ return find_format(0, code, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_find_mbus_format);
- if (index >= ARRAY_SIZE(imx_media_formats))
- return -EINVAL;
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+ bool allow_bayer)
+{
+ return enum_format(NULL, code, index, cs_sel, false, allow_bayer);
+}
+EXPORT_SYMBOL_GPL(imx_media_enum_mbus_format);
- fmt = &imx_media_formats[index];
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel)
+{
+ const struct imx_media_pixfmt *array, *fmt, *ret = NULL;
+ u32 array_size;
+ int i, j;
- if ((fmt->cs == IPUV3_COLORSPACE_RGB && !allow_rgb) ||
- (fmt->planar && !allow_planar))
- return -EINVAL;
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ array_size = NUM_IPU_YUV_FORMATS;
+ array = ipu_yuv_formats;
+ break;
+ case CS_SEL_RGB:
+ array_size = NUM_IPU_RGB_FORMATS;
+ array = ipu_rgb_formats;
+ break;
+ case CS_SEL_ANY:
+ array_size = NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS;
+ array = ipu_yuv_formats;
+ break;
+ default:
+ return NULL;
+ }
- if (code)
- *code = fmt->codes[0];
- if (fourcc)
- *fourcc = fmt->fourcc;
+ for (i = 0; i < array_size; i++) {
+ if (cs_sel == CS_SEL_ANY && i >= NUM_IPU_YUV_FORMATS)
+ fmt = &ipu_rgb_formats[i - NUM_IPU_YUV_FORMATS];
+ else
+ fmt = &array[i];
- return 0;
+ for (j = 0; code && fmt->codes[j]; j++) {
+ if (code == fmt->codes[j]) {
+ ret = fmt;
+ goto out;
+ }
+ }
+ }
+
+out:
+ }
+ }
+ }
+
+ return ret;
}
-EXPORT_SYMBOL_GPL(imx_media_enum_format);
+EXPORT_SYMBOL_GPL(imx_media_find_ipu_format);
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb)
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel)
{
- const struct imx_media_pixfmt *fmt;
- u32 lcode;
-
- if (index >= ARRAY_SIZE(imx_media_ipu_internal_codes))
- return -EINVAL;
-
- lcode = imx_media_ipu_internal_codes[index];
-
- fmt = find_format(0, lcode, allow_rgb, false, true);
- if (!fmt)
+ switch (cs_sel) {
+ case CS_SEL_YUV:
+ if (index >= NUM_IPU_YUV_FORMATS)
+ return -EINVAL;
+ *code = ipu_yuv_formats[index].codes[0];
+ break;
+ case CS_SEL_RGB:
+ if (index >= NUM_IPU_RGB_FORMATS)
+ return -EINVAL;
+ *code = ipu_rgb_formats[index].codes[0];
+ break;
+ case CS_SEL_ANY:
+ if (index >= NUM_IPU_YUV_FORMATS + NUM_IPU_RGB_FORMATS)
+ return -EINVAL;
+ if (index >= NUM_IPU_YUV_FORMATS) {
+ index -= NUM_IPU_YUV_FORMATS;
+ *code = ipu_rgb_formats[index].codes[0];
+ } else {
+ *code = ipu_yuv_formats[index].codes[0];
+ }
+ break;
+ default:
return -EINVAL;
+ break;
+ default:
-
- if (code)
- *code = fmt->codes[0];
- if (fourcc)
- *fourcc = fmt->fourcc;
+ }
return 0;
}
@@ -283,10 +440,14 @@ int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
mbus->height = height;
mbus->field = field;
if (code == 0)
- imx_media_enum_format(NULL, &code, 0, true, false);
- lcc = imx_media_find_format(0, code, true, false);
- if (!lcc)
- return -EINVAL;
+ imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+ lcc = imx_media_find_mbus_format(code, CS_SEL_ANY, false);
+ if (!lcc) {
+ lcc = imx_media_find_ipu_format(code, CS_SEL_ANY);
+ if (!lcc)
+ return -EINVAL;
+ }
+
mbus->code = code;
+ }
+
mbus->colorspace = pixfmt_to_colorspace(lcc);
@@ -304,11 +465,25 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
u32 stride;
if (!cc) {
- cc = imx_media_find_format(0, mbus->code, true, false);
+ cc = imx_media_find_ipu_format(mbus->code, CS_SEL_ANY);
+ if (!cc)
+ cc = imx_media_find_mbus_format(mbus->code, CS_SEL_ANY,
+ true);
if (!cc)
return -EINVAL;
}
+ /*
+ * TODO: the IPU currently does not support the AYUV32 format,
+ * so until it does convert to a supported YUV format.
+ */
+ if (cc->ipufmt && cc->cs == IPUV3_COLORSPACE_YUV) {
+ u32 code;
+
+ imx_media_enum_mbus_format(&code, 0, CS_SEL_YUV, false);
+ cc = imx_media_find_mbus_format(code, CS_SEL_YUV, false);
+ }
+
stride = cc->planar ? mbus->width : (mbus->width * cc->bpp) >> 3;
pix->width = mbus->width;
@@ -349,7 +524,7 @@ int imx_media_ipu_image_to_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
{
const struct imx_media_pixfmt *fmt;
- fmt = imx_media_find_format(image->pix.pixelformat, 0, true, false);
+ fmt = imx_media_find_format(image->pix.pixelformat, CS_SEL_ANY);
if (!fmt)
return -EINVAL;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 0e7c24c..070f9da 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -517,20 +517,6 @@ static int vdic_s_stream(struct v4l2_subdev *sd, int enable)
return ret;
}
-static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- if (code->pad >= VDIC_NUM_PADS)
- return -EINVAL;
-
- if (code->pad == VDIC_SINK_PAD_IDMAC)
- return imx_media_enum_format(NULL, &code->code, code->index,
- false, false);
-
- return imx_media_enum_ipu_format(NULL, &code->code, code->index, false);
-}
-
static struct v4l2_mbus_framefmt *
__vdic_get_fmt(struct vdic_priv *priv, struct v4l2_subdev_pad_config *cfg,
unsigned int pad, enum v4l2_subdev_format_whence which)
@@ -541,6 +527,16 @@ __vdic_get_fmt(struct vdic_priv *priv, struct v4l2_subdev_pad_config *cfg,
return &priv->format_mbus[pad];
}
+static int vdic_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad >= VDIC_NUM_PADS)
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ return -EINVAL;
+
+ return imx_media_enum_ipu_format(&code->code, code->index, CS_SEL_YUV);
+}
+
static int vdic_get_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat)
@@ -586,49 +582,28 @@ static int vdic_set_fmt(struct v4l2_subdev *sd,
goto out;
}
- v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W_VDIC,
- W_ALIGN, &sdformat->format.height,
- MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
+ cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_YUV);
+ if (!cc) {
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
+ cc = imx_media_find_ipu_format(code, CS_SEL_YUV);
+ sdformat->format.code = cc->codes[0];
+ }
switch (sdformat->pad) {
+ }
case VDIC_SRC_PAD_DIRECT:
infmt = __vdic_get_fmt(priv, cfg, priv->active_input_pad,
sdformat->which);
-
- cc = imx_media_find_ipu_format(0, sdformat->format.code, false);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0, false);
- cc = imx_media_find_ipu_format(0, code, false);
- sdformat->format.code = cc->codes[0];
- }
-
sdformat->format.width = infmt->width;
sdformat->format.height = infmt->height;
/* output is always progressive! */
sdformat->format.field = V4L2_FIELD_NONE;
break;
- case VDIC_SINK_PAD_IDMAC:
case VDIC_SINK_PAD_DIRECT:
- if (sdformat->pad == VDIC_SINK_PAD_DIRECT) {
- cc = imx_media_find_ipu_format(0, sdformat->format.code,
- false);
- if (!cc) {
- imx_media_enum_ipu_format(NULL, &code, 0,
- false);
- cc = imx_media_find_ipu_format(0, code, false);
- sdformat->format.code = cc->codes[0];
- }
- } else {
- cc = imx_media_find_format(0, sdformat->format.code,
- false, false);
- if (!cc) {
- imx_media_enum_format(NULL, &code, 0,
- false, false);
- cc = imx_media_find_format(0, code,
- false, false);
- sdformat->format.code = cc->codes[0];
- }
- }
+ case VDIC_SINK_PAD_IDMAC:
+ v4l_bound_align_image(&sdformat->format.width,
+ MIN_W, MAX_W_VDIC, W_ALIGN,
+ &sdformat->format.height,
+ MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN);
/* input must be interlaced! Choose SEQ_TB if not */
if (!V4L2_FIELD_HAS_BOTH(sdformat->format.field))
@@ -814,7 +789,7 @@ static int vdic_registered(struct v4l2_subdev *sd)
code = 0;
if (i != VDIC_SINK_PAD_IDMAC)
- imx_media_enum_ipu_format(NULL, &code, 0, true);
+ imx_media_enum_ipu_format(&code, 0, CS_SEL_YUV);
/* set a default mbus format */
ret = imx_media_init_mbus_fmt(&priv->format_mbus[i],
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index a67ee14..2342422 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -171,16 +171,23 @@ struct imx_media_dev {
struct v4l2_async_notifier subdev_notifier;
};
-const struct imx_media_pixfmt *imx_media_find_format(u32 fourcc, u32 code,
- bool allow_rgb,
- bool allow_planar);
-const struct imx_media_pixfmt *imx_media_find_ipu_format(u32 fourcc, u32 code,
- bool allow_rgb);
-
-int imx_media_enum_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb, bool allow_planar);
-int imx_media_enum_ipu_format(u32 *fourcc, u32 *code, u32 index,
- bool allow_rgb);
+enum codespace_sel {
+ CS_SEL_YUV = 0,
+ CS_SEL_RGB,
+ CS_SEL_ANY,
+};
+
+const struct imx_media_pixfmt *
+imx_media_find_format(u32 fourcc, enum codespace_sel cs_sel);
+int imx_media_enum_format(u32 *fourcc, u32 index, enum codespace_sel cs_sel);
+const struct imx_media_pixfmt *
+imx_media_find_mbus_format(u32 code, enum codespace_sel cs_sel,
+ bool allow_bayer);
+int imx_media_enum_mbus_format(u32 *code, u32 index, enum codespace_sel cs_sel,
+ bool allow_bayer);
+const struct imx_media_pixfmt *
+imx_media_find_ipu_format(u32 code, enum codespace_sel cs_sel);
+int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);
int imx_media_init_mbus_fmt(struct v4l2_mbus_framefmt *mbus,
u32 width, u32 height, u32 code, u32 field,
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
The sabreauto uses a steering pin to select between the SDA signal on
i2c3 bus, and a data-in pin for an SPI NOR chip. Use i2cmux to control
this steering pin. Idle state of the i2cmux selects SPI NOR. This is not
a classic way to use i2cmux, since one side of the mux selects something
other than an i2c bus, but it works and is probably the cleanest
solution. Note that if one thread is attempting to access SPI NOR while
another thread is accessing i2c3, the SPI NOR access will fail since the
i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't
be avoided in any case, the board is not designed to allow concurrent
i2c3 and SPI NOR functions (and the default device-tree does not enable
SPI NOR anyway).
Devices hanging off i2c3 should now be defined under i2cmux, so
that the steering pin can be properly controlled to access those
devices. The port expanders (MAX7310) are thus moved into i2cmux.
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 65 +++++++++++++++++++++-----------
1 file changed, 44 insertions(+), 21 deletions(-)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index a2a714d..c8e35c4 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -108,6 +108,44 @@
default-brightness-level = <7>;
status = "okay";
};
+
+ i2cmux {
+ compatible = "i2c-mux-gpio";
+ pinctrl-0 = <&pinctrl_i2c3mux>;
+ mux-gpios = <&gpio5 4 0>;
+ i2c-parent = <&i2c3>;
+ idle-state = <0>;
+
+ i2c@1 {
+ compatible = "maxim,max7310";
+ reg = <0x30>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+
+ max7310_b: gpio@32 {
+ compatible = "maxim,max7310";
+ reg = <0x32>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+
+ max7310_c: gpio@34 {
+ compatible = "maxim,max7310";
+ reg = <0x34>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+ };
+ };
};
&clks {
@@ -290,27 +328,6 @@
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c3>;
status = "okay";
-
- max7310_a: gpio@30 {
- compatible = "maxim,max7310";
- reg = <0x30>;
- gpio-controller;
- #gpio-cells = <2>;
- };
-
- max7310_b: gpio@32 {
- compatible = "maxim,max7310";
- reg = <0x32>;
- gpio-controller;
- #gpio-cells = <2>;
- };
-
- max7310_c: gpio@34 {
- compatible = "maxim,max7310";
- reg = <0x34>;
- gpio-controller;
- #gpio-cells = <2>;
- };
};
&iomuxc {
@@ -418,6 +435,12 @@
>;
};
+ pinctrl_i2c3mux: i2c3muxgrp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_A24__GPIO5_IO04 0x0b0b1
+ >;
+ };
+
pinctrl_pwm3: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x1b0b1
--
2.7.4
i2c3 bus, and a data-in pin for an SPI NOR chip. Use i2cmux to control
this steering pin. Idle state of the i2cmux selects SPI NOR. This is not
a classic way to use i2cmux, since one side of the mux selects something
other than an i2c bus, but it works and is probably the cleanest
solution. Note that if one thread is attempting to access SPI NOR while
another thread is accessing i2c3, the SPI NOR access will fail since the
i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't
be avoided in any case, the board is not designed to allow concurrent
i2c3 and SPI NOR functions (and the default device-tree does not enable
SPI NOR anyway).
Devices hanging off i2c3 should now be defined under i2cmux, so
that the steering pin can be properly controlled to access those
devices. The port expanders (MAX7310) are thus moved into i2cmux.
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 65 +++++++++++++++++++++-----------
1 file changed, 44 insertions(+), 21 deletions(-)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index a2a714d..c8e35c4 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -108,6 +108,44 @@
default-brightness-level = <7>;
status = "okay";
};
+
+ i2cmux {
+ compatible = "i2c-mux-gpio";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default";
+ #size-cells = <0>;
+ pinctrl-0 = <&pinctrl_i2c3mux>;
+ mux-gpios = <&gpio5 4 0>;
+ i2c-parent = <&i2c3>;
+ idle-state = <0>;
+
+ i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ #size-cells = <0>;
+ reg = <1>;
+
+ max7310_a: gpio@30 {
+
+ compatible = "maxim,max7310";
+ reg = <0x30>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+
+ max7310_b: gpio@32 {
+ compatible = "maxim,max7310";
+ reg = <0x32>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+
+ max7310_c: gpio@34 {
+ compatible = "maxim,max7310";
+ reg = <0x34>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ };
+ };
+ };
};
&clks {
@@ -290,27 +328,6 @@
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c3>;
status = "okay";
-
- max7310_a: gpio@30 {
- compatible = "maxim,max7310";
- reg = <0x30>;
- gpio-controller;
- #gpio-cells = <2>;
- };
-
- max7310_b: gpio@32 {
- compatible = "maxim,max7310";
- reg = <0x32>;
- gpio-controller;
- #gpio-cells = <2>;
- };
-
- max7310_c: gpio@34 {
- compatible = "maxim,max7310";
- reg = <0x34>;
- gpio-controller;
- #gpio-cells = <2>;
- };
};
&iomuxc {
@@ -418,6 +435,12 @@
>;
};
+ pinctrl_i2c3mux: i2c3muxgrp {
+ fsl,pins = <
+ MX6QDL_PAD_EIM_A24__GPIO5_IO04 0x0b0b1
+ >;
+ };
+
pinctrl_pwm3: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x1b0b1
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
output device has measured an interval between the reception or transmit
completion of two consecutive frames of video that is outside the nominal
frame interval by some tolerance value.
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 6 ++++++
Documentation/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 8 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
index 8d663a7..dc77363 100644
--- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
+++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
@@ -197,6 +197,12 @@ call.
the regions changes. This event has a struct
:c:type:`v4l2_event_motion_det`
associated with it.
+ * - ``V4L2_EVENT_FRAME_INTERVAL_ERROR``
+ - 7
+ - This event is triggered when the video capture or output device
+ has measured an interval between the reception or transmit
+ completion of two consecutive frames of video that is outside
+ the nominal frame interval by some tolerance value.
* - ``V4L2_EVENT_PRIVATE_START``
- 0x08000000
- Base event number for driver-private events.
diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
index e11a0d0..c7d8fad 100644
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -459,6 +459,7 @@ replace define V4L2_EVENT_CTRL event-type
replace define V4L2_EVENT_FRAME_SYNC event-type
replace define V4L2_EVENT_SOURCE_CHANGE event-type
replace define V4L2_EVENT_MOTION_DET event-type
+replace define V4L2_EVENT_FRAME_INTERVAL_ERROR event-type
replace define V4L2_EVENT_PRIVATE_START event-type
replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 45184a2..cf5a0d0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2131,6 +2131,7 @@ struct v4l2_streamparm {
#define V4L2_EVENT_FRAME_SYNC 4
#define V4L2_EVENT_SOURCE_CHANGE 5
#define V4L2_EVENT_MOTION_DET 6
+#define V4L2_EVENT_FRAME_INTERVAL_ERROR 7
#define V4L2_EVENT_PRIVATE_START 0x08000000
/* Payload for V4L2_EVENT_VSYNC */
--
2.7.4
output device has measured an interval between the reception or transmit
completion of two consecutive frames of video that is outside the nominal
frame interval by some tolerance value.
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 6 ++++++
Documentation/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 8 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
index 8d663a7..dc77363 100644
--- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
+++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
@@ -197,6 +197,12 @@ call.
the regions changes. This event has a struct
:c:type:`v4l2_event_motion_det`
associated with it.
+ * - ``V4L2_EVENT_FRAME_INTERVAL_ERROR``
+ - 7
+ - This event is triggered when the video capture or output device
+ has measured an interval between the reception or transmit
+ completion of two consecutive frames of video that is outside
+ the nominal frame interval by some tolerance value.
* - ``V4L2_EVENT_PRIVATE_START``
- 0x08000000
- Base event number for driver-private events.
diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
index e11a0d0..c7d8fad 100644
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -459,6 +459,7 @@ replace define V4L2_EVENT_CTRL event-type
replace define V4L2_EVENT_FRAME_SYNC event-type
replace define V4L2_EVENT_SOURCE_CHANGE event-type
replace define V4L2_EVENT_MOTION_DET event-type
+replace define V4L2_EVENT_FRAME_INTERVAL_ERROR event-type
replace define V4L2_EVENT_PRIVATE_START event-type
replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 45184a2..cf5a0d0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2131,6 +2131,7 @@ struct v4l2_streamparm {
#define V4L2_EVENT_FRAME_SYNC 4
#define V4L2_EVENT_SOURCE_CHANGE 5
#define V4L2_EVENT_MOTION_DET 6
+#define V4L2_EVENT_FRAME_INTERVAL_ERROR 7
#define V4L2_EVENT_PRIVATE_START 0x08000000
/* Payload for V4L2_EVENT_VSYNC */
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
From: Russell King <rmk+k...@armlinux.org.uk>
Setting and getting frame rates is part of the negotiation mechanism
between subdevs. The lack of support means that a frame rate at the
sensor can't be negotiated through the subdev path.
Add support at MIPI CSI2 level for handling this part of the
negotiation.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
drivers/staging/media/imx/imx6-mipi-csi2.c | 36 ++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 1a71b40..d8f931e 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -49,6 +49,7 @@ struct csi2_dev {
struct mutex lock;
struct v4l2_mbus_framefmt format_mbus;
+ struct v4l2_fract frame_interval;
int power_count;
bool stream_on;
@@ -487,6 +488,35 @@ static int csi2_set_fmt(struct v4l2_subdev *sd,
return ret;
}
+static int csi2_g_frame_interval(struct v4l2_subdev *sd,
+ fi->interval = csi2->frame_interval;
+ mutex_unlock(&csi2->lock);
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad != CSI2_SINK_PAD)
+ fi->interval = csi2->frame_interval;
+
+ csi2->frame_interval = fi->interval;
+
+ mutex_unlock(&csi2->lock);
struct media_link *link,
struct v4l2_subdev_format *source_fmt,
@@ -535,6 +565,8 @@ static struct v4l2_subdev_core_ops csi2_core_ops = {
static struct v4l2_subdev_video_ops csi2_video_ops = {
.s_stream = csi2_s_stream,
+ .g_frame_interval = csi2_g_frame_interval,
+ .s_frame_interval = csi2_s_frame_interval,
};
static struct v4l2_subdev_pad_ops csi2_pad_ops = {
@@ -603,6 +635,10 @@ static int csi2_probe(struct platform_device *pdev)
csi2->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
csi2->sd.grp_id = IMX_MEDIA_GRP_ID_CSI2;
+ /* init default frame interval */
+ csi2->frame_interval.numerator = 1;
+ csi2->frame_interval.denominator = 30;
+
ret = csi2_parse_endpoints(csi2);
if (ret)
return ret;
--
2.7.4
Setting and getting frame rates is part of the negotiation mechanism
between subdevs. The lack of support means that a frame rate at the
sensor can't be negotiated through the subdev path.
Add support at MIPI CSI2 level for handling this part of the
negotiation.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
drivers/staging/media/imx/imx6-mipi-csi2.c | 36 ++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 1a71b40..d8f931e 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -49,6 +49,7 @@ struct csi2_dev {
struct mutex lock;
struct v4l2_mbus_framefmt format_mbus;
+ struct v4l2_fract frame_interval;
int power_count;
bool stream_on;
@@ -487,6 +488,35 @@ static int csi2_set_fmt(struct v4l2_subdev *sd,
return ret;
}
+static int csi2_g_frame_interval(struct v4l2_subdev *sd,
+ struct v4l2_subdev_frame_interval *fi)
+{
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+
+ mutex_lock(&csi2->lock);
+
+ fi->interval = csi2->frame_interval;
+ mutex_unlock(&csi2->lock);
+
+ return 0;
+}
+
+static int csi2_s_frame_interval(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+
+ mutex_lock(&csi2->lock);
+
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad != CSI2_SINK_PAD)
+ fi->interval = csi2->frame_interval;
+
+ csi2->frame_interval = fi->interval;
+
+ mutex_unlock(&csi2->lock);
+ return 0;
+}
+
static int csi2_link_validate(struct v4l2_subdev *sd,
+}
+
struct media_link *link,
struct v4l2_subdev_format *source_fmt,
@@ -535,6 +565,8 @@ static struct v4l2_subdev_core_ops csi2_core_ops = {
static struct v4l2_subdev_video_ops csi2_video_ops = {
.s_stream = csi2_s_stream,
+ .g_frame_interval = csi2_g_frame_interval,
+ .s_frame_interval = csi2_s_frame_interval,
};
static struct v4l2_subdev_pad_ops csi2_pad_ops = {
@@ -603,6 +635,10 @@ static int csi2_probe(struct platform_device *pdev)
csi2->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
csi2->sd.grp_id = IMX_MEDIA_GRP_ID_CSI2;
+ /* init default frame interval */
+ csi2->frame_interval.denominator = 30;
+
ret = csi2_parse_endpoints(csi2);
if (ret)
return ret;
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:06 AM3/10/17
to
The reset pin to the port expander chip (MAX7310) is controlled by a gpio,
so define a reset-gpios property to control it. There are three MAX7310's
on the SabreAuto CPU card (max7310_[abc]), but all use the same pin for
their reset. Since all can't acquire the same pin, assign it to max7310_b,
that chip is needed by more functions (usb and adv7180).
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index c8e35c4..21dea5f 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -136,6 +136,9 @@
reg = <0x32>;
gpio-controller;
#gpio-cells = <2>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_max7310>;
+ reset-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>;
};
max7310_c: gpio@34 {
@@ -441,6 +444,12 @@
>;
};
+ pinctrl_max7310: max7310grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
so define a reset-gpios property to control it. There are three MAX7310's
on the SabreAuto CPU card (max7310_[abc]), but all use the same pin for
their reset. Since all can't acquire the same pin, assign it to max7310_b,
that chip is needed by more functions (usb and adv7180).
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index c8e35c4..21dea5f 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -136,6 +136,9 @@
reg = <0x32>;
gpio-controller;
#gpio-cells = <2>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_max7310>;
+ reset-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>;
};
max7310_c: gpio@34 {
@@ -441,6 +444,12 @@
>;
};
+ pinctrl_max7310: max7310grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
Steve Longerbeam
Mar 10, 2017, 12:00:07 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
This patch adds the device tree graph connecting the input multiplexers
to the IPU CSIs and the MIPI-CSI2 gasket on i.MX6. The MIPI_IPU
multiplexers are added as children of the iomuxc-gpr syscon device node.
On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2
CSI1 allow to select between CSI0/1 parallel input pads and the MIPI
CSI-2 virtual channels 0/3.
On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1
CSI1 allow to select between CSI0/1 parallel input pads and any of the
four MIPI CSI-2 virtual channels.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
--
- Removed some dangling/unused endpoints (ipu2_csi0_from_csi2ipu)
- Renamed the mipi virtual channel endpoint labels, from "mipi_csiX_..."
to "mipi_vcX...".
- Added input endpoint anchors to the video muxes for the connections
from parallel sensors.
arch/arm/boot/dts/imx6dl.dtsi | 180 +++++++++++++++++++++++++++++++++++++++++
arch/arm/boot/dts/imx6q.dtsi | 116 ++++++++++++++++++++++++++
arch/arm/boot/dts/imx6qdl.dtsi | 10 ++-
3 files changed, 305 insertions(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 7aa120f..8958c4a 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -181,6 +181,186 @@
"di0", "di1";
};
+&gpr {
+ ipu1_csi0_mux: ipu1_csi0_mux@34 {
+ compatible = "video-multiplexer";
+ reg = <0x34>;
+ bit-mask = <0x7>;
+ bit-shift = <0>;
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi0_mux>;
+ remote-endpoint = <&mipi_vc2_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ ipu1_csi0_mux_from_mipi_vc3: endpoint {
+ remote-endpoint = <&mipi_vc3_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ ipu1_csi0_mux_from_parallel_sensor: endpoint {
+ };
+ };
+
+ port@5 {
+ reg = <5>;
+
+ ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+ remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+ };
+ };
+ };
+
+ ipu1_csi1_mux: ipu1_csi1_mux@34 {
+ compatible = "video-multiplexer";
+ reg = <0x34>;
+ bit-mask = <0x7>;
+ bit-shift = <3>;
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi1_mux>;
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi1_mux>;
+ remote-endpoint = <&mipi_vc2_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ ipu1_csi1_mux_from_mipi_vc3: endpoint {
+ remote-endpoint = <&mipi_vc3_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ ipu1_csi1_mux_from_parallel_sensor: endpoint {
+ };
+ };
+
+ port@5 {
+ reg = <5>;
+
+ ipu1_csi1_mux_to_ipu1_csi1: endpoint {
+ remote-endpoint = <&ipu1_csi1_from_ipu1_csi1_mux>;
+ ipu1_csi1_from_ipu1_csi1_mux: endpoint {
+ remote-endpoint = <&ipu1_csi1_mux_to_ipu1_csi1>;
+ };
+};
+
+&mipi_csi {
+ port@1 {
+ reg = <1>;
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+ };
+
+ mipi_vc0_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc0>;
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc1>;
+ };
+
+ mipi_vc1_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc1>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc2>;
+ };
+
+ mipi_vc2_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc2>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc3>;
+ };
+
+ mipi_vc3_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc3>;
+ };
+ };
+};
+
&vpu {
compatible = "fsl,imx6dl-vpu", "cnm,coda960";
};
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index e9a5d0b..b833b0d 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -143,10 +143,18 @@
ipu2_csi0: port@0 {
reg = <0>;
+
+ ipu2_csi0_from_mipi_vc2: endpoint {
+ remote-endpoint = <&mipi_vc2_to_ipu2_csi0>;
+ };
};
ipu2_csi1: port@1 {
reg = <1>;
+
+ ipu2_csi1_from_ipu2_csi1_mux: endpoint {
+ remote-endpoint = <&ipu2_csi1_mux_to_ipu2_csi1>;
+ };
};
ipu2_di0: port@2 {
@@ -266,6 +274,80 @@
};
};
+&gpr {
+ ipu1_csi0_mux: ipu1_csi0_mux@4 {
+ compatible = "video-multiplexer";
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+ remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+ };
+ };
+ };
+
+ ipu2_csi1_mux: ipu2_csi1_mux@4 {
+ compatible = "video-multiplexer";
+
+ remote-endpoint = <&mipi_vc3_to_ipu2_csi1_mux>;
+ remote-endpoint = <&ipu2_csi1_from_ipu2_csi1_mux>;
+ ipu1_csi1_from_mipi_vc1: endpoint {
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi1>;
+ };
+};
+
&ldb {
clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>, <&clks IMX6QDL_CLK_LDB_DI1_SEL>,
<&clks IMX6QDL_CLK_IPU1_DI0_SEL>, <&clks IMX6QDL_CLK_IPU1_DI1_SEL>,
@@ -312,6 +394,40 @@
};
};
+&mipi_csi {
+ port@1 {
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+ remote-endpoint = <&ipu1_csi1_from_mipi_vc1>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ mipi_vc2_to_ipu2_csi0: endpoint {
+ remote-endpoint = <&ipu2_csi0_from_mipi_vc2>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ mipi_vc3_to_ipu2_csi1_mux: endpoint {
+ remote-endpoint = <&ipu2_csi1_mux_from_mipi_vc3>;
+ };
+ };
+};
+
&mipi_dsi {
ports {
port@2 {
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index d28a413..194badd 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -807,8 +807,10 @@
};
gpr: iomuxc-gpr@020e0000 {
- compatible = "fsl,imx6q-iomuxc-gpr", "syscon";
+ compatible = "fsl,imx6q-iomuxc-gpr", "syscon", "simple-mfd";
reg = <0x020e0000 0x38>;
iomuxc: iomuxc@020e0000 {
@@ -1136,6 +1138,8 @@
mipi_csi: mipi@021dc000 {
compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
reg = <0x021dc000 0x4000>;
clocks = <&clks IMX6QDL_CLK_HSI_TX>,
<&clks IMX6QDL_CLK_VIDEO_27M>,
@@ -1243,6 +1247,10 @@
ipu1_csi0: port@0 {
reg = <0>;
+
+ ipu1_csi0_from_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_to_ipu1_csi0>;
+ };
};
ipu1_csi1: port@1 {
--
2.7.4
This patch adds the device tree graph connecting the input multiplexers
to the IPU CSIs and the MIPI-CSI2 gasket on i.MX6. The MIPI_IPU
multiplexers are added as children of the iomuxc-gpr syscon device node.
On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2
CSI1 allow to select between CSI0/1 parallel input pads and the MIPI
CSI-2 virtual channels 0/3.
On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1
CSI1 allow to select between CSI0/1 parallel input pads and any of the
four MIPI CSI-2 virtual channels.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
--
- Removed some dangling/unused endpoints (ipu2_csi0_from_csi2ipu)
- Renamed the mipi virtual channel endpoint labels, from "mipi_csiX_..."
to "mipi_vcX...".
- Added input endpoint anchors to the video muxes for the connections
from parallel sensors.
arch/arm/boot/dts/imx6dl.dtsi | 180 +++++++++++++++++++++++++++++++++++++++++
arch/arm/boot/dts/imx6q.dtsi | 116 ++++++++++++++++++++++++++
arch/arm/boot/dts/imx6qdl.dtsi | 10 ++-
3 files changed, 305 insertions(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 7aa120f..8958c4a 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -181,6 +181,186 @@
"di0", "di1";
};
+&gpr {
+ ipu1_csi0_mux: ipu1_csi0_mux@34 {
+ compatible = "video-multiplexer";
+ reg = <0x34>;
+ bit-mask = <0x7>;
+ bit-shift = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ ipu1_csi0_mux_from_mipi_vc0: endpoint {
+ reg = <0>;
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@1 {
+ };
+
+ port@1 {
+ reg = <1>;
+
+ ipu1_csi0_mux_from_mipi_vc1: endpoint {
+
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ ipu1_csi0_mux_from_mipi_vc2: endpoint {
+ };
+
+ port@2 {
+ reg = <2>;
+
+ remote-endpoint = <&mipi_vc2_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ ipu1_csi0_mux_from_mipi_vc3: endpoint {
+ remote-endpoint = <&mipi_vc3_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ ipu1_csi0_mux_from_parallel_sensor: endpoint {
+ };
+ };
+
+ port@5 {
+ reg = <5>;
+
+ ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+ remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+ };
+ };
+ };
+
+ ipu1_csi1_mux: ipu1_csi1_mux@34 {
+ compatible = "video-multiplexer";
+ reg = <0x34>;
+ bit-mask = <0x7>;
+ bit-shift = <3>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ ipu1_csi1_mux_from_mipi_vc0: endpoint {
+ reg = <0>;
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@1 {
+ };
+
+ port@1 {
+ reg = <1>;
+
+ ipu1_csi1_mux_from_mipi_vc1: endpoint {
+
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ ipu1_csi1_mux_from_mipi_vc2: endpoint {
+ };
+
+ port@2 {
+ reg = <2>;
+
+ remote-endpoint = <&mipi_vc2_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ ipu1_csi1_mux_from_mipi_vc3: endpoint {
+ remote-endpoint = <&mipi_vc3_to_ipu1_csi1_mux>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ ipu1_csi1_mux_from_parallel_sensor: endpoint {
+ };
+ };
+
+ port@5 {
+ reg = <5>;
+
+ ipu1_csi1_mux_to_ipu1_csi1: endpoint {
+ remote-endpoint = <&ipu1_csi1_from_ipu1_csi1_mux>;
+ };
+ };
+ };
+};
+
+&ipu1_csi1 {
+ };
+ };
+};
+
+ ipu1_csi1_from_ipu1_csi1_mux: endpoint {
+ remote-endpoint = <&ipu1_csi1_mux_to_ipu1_csi1>;
+ };
+};
+
+&mipi_csi {
+ port@1 {
+ reg = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mipi_vc0_to_ipu1_csi0_mux: endpoint@0 {
+ #size-cells = <0>;
+
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+ };
+
+ mipi_vc0_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc0>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+ };
+
+ port@2 {
+ reg = <2>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mipi_vc1_to_ipu1_csi0_mux: endpoint@0 {
+ #size-cells = <0>;
+
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc1>;
+ };
+
+ mipi_vc1_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc1>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mipi_vc2_to_ipu1_csi0_mux: endpoint@0 {
+ #size-cells = <0>;
+
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc2>;
+ };
+
+ mipi_vc2_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc2>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mipi_vc3_to_ipu1_csi0_mux: endpoint@0 {
+ #size-cells = <0>;
+
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc3>;
+ };
+
+ mipi_vc3_to_ipu1_csi1_mux: endpoint@1 {
+ remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc3>;
+ };
+ };
+};
+
&vpu {
compatible = "fsl,imx6dl-vpu", "cnm,coda960";
};
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index e9a5d0b..b833b0d 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -143,10 +143,18 @@
ipu2_csi0: port@0 {
reg = <0>;
+
+ ipu2_csi0_from_mipi_vc2: endpoint {
+ remote-endpoint = <&mipi_vc2_to_ipu2_csi0>;
+ };
};
ipu2_csi1: port@1 {
reg = <1>;
+
+ ipu2_csi1_from_ipu2_csi1_mux: endpoint {
+ remote-endpoint = <&ipu2_csi1_mux_to_ipu2_csi1>;
+ };
};
ipu2_di0: port@2 {
@@ -266,6 +274,80 @@
};
};
+&gpr {
+ ipu1_csi0_mux: ipu1_csi0_mux@4 {
+ compatible = "video-multiplexer";
+ reg = <0x04>;
+ bit-mask = <1>;
+ bit-shift = <19>;
+ bit-mask = <1>;
+ bit-shift = <19>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ ipu1_csi0_mux_from_mipi_vc0: endpoint {
+ reg = <0>;
+
+ remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+ };
+ };
+
+ port@1 {
+ };
+
+ port@1 {
+ reg = <1>;
+
+ ipu1_csi0_mux_from_parallel_sensor: endpoint {
+
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+ };
+
+ port@2 {
+ reg = <2>;
+
+ remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+ };
+ };
+ };
+
+ ipu2_csi1_mux: ipu2_csi1_mux@4 {
+ compatible = "video-multiplexer";
+ reg = <0x04>;
+ bit-mask = <1>;
+ bit-shift = <20>;
+ bit-mask = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "okay";
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ ipu2_csi1_mux_from_mipi_vc3: endpoint {
+ reg = <0>;
+
+ remote-endpoint = <&mipi_vc3_to_ipu2_csi1_mux>;
+ };
+ };
+
+ port@1 {
+ };
+
+ port@1 {
+ reg = <1>;
+
+ ipu2_csi1_mux_from_parallel_sensor: endpoint {
+
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ ipu2_csi1_mux_to_ipu2_csi1: endpoint {
+ };
+
+ port@2 {
+ reg = <2>;
+
+ remote-endpoint = <&ipu2_csi1_from_ipu2_csi1_mux>;
+ };
+ };
+ };
+};
+
+&ipu1_csi1 {
+ };
+ };
+};
+
+ ipu1_csi1_from_mipi_vc1: endpoint {
+ remote-endpoint = <&mipi_vc1_to_ipu1_csi1>;
+ };
+};
+
&ldb {
clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>, <&clks IMX6QDL_CLK_LDB_DI1_SEL>,
<&clks IMX6QDL_CLK_IPU1_DI0_SEL>, <&clks IMX6QDL_CLK_IPU1_DI1_SEL>,
@@ -312,6 +394,40 @@
};
};
+&mipi_csi {
+ port@1 {
+ reg = <1>;
+
+ mipi_vc0_to_ipu1_csi0_mux: endpoint {
+
+ remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+ };
+ };
+
+ port@2 {
+ reg = <2>;
+
+ mipi_vc1_to_ipu1_csi1: endpoint {
+ };
+
+ port@2 {
+ reg = <2>;
+
+ remote-endpoint = <&ipu1_csi1_from_mipi_vc1>;
+ };
+ };
+
+ port@3 {
+ reg = <3>;
+
+ mipi_vc2_to_ipu2_csi0: endpoint {
+ remote-endpoint = <&ipu2_csi0_from_mipi_vc2>;
+ };
+ };
+
+ port@4 {
+ reg = <4>;
+
+ mipi_vc3_to_ipu2_csi1_mux: endpoint {
+ remote-endpoint = <&ipu2_csi1_mux_from_mipi_vc3>;
+ };
+ };
+};
+
&mipi_dsi {
ports {
port@2 {
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index d28a413..194badd 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -807,8 +807,10 @@
};
gpr: iomuxc-gpr@020e0000 {
- compatible = "fsl,imx6q-iomuxc-gpr", "syscon";
+ compatible = "fsl,imx6q-iomuxc-gpr", "syscon", "simple-mfd";
reg = <0x020e0000 0x38>;
+ #address-cells = <1>;
+ #size-cells = <0>;
};
+ #size-cells = <0>;
iomuxc: iomuxc@020e0000 {
@@ -1136,6 +1138,8 @@
mipi_csi: mipi@021dc000 {
compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
reg = <0x021dc000 0x4000>;
+ #address-cells = <1>;
+ #size-cells = <0>;
interrupts = <0 100 0x04>, <0 101 0x04>;
+ #size-cells = <0>;
clocks = <&clks IMX6QDL_CLK_HSI_TX>,
<&clks IMX6QDL_CLK_VIDEO_27M>,
@@ -1243,6 +1247,10 @@
ipu1_csi0: port@0 {
reg = <0>;
+
+ ipu1_csi0_from_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_to_ipu1_csi0>;
+ };
};
ipu1_csi1: port@1 {
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:07 AM3/10/17
to
Add device tree binding documentation for the OV5640 camera sensor.
.../devicetree/bindings/media/i2c/ov5640.txt | 45 ++++++++++++++++++++++
1 file changed, 45 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
diff --git a/Documentation/devicetree/bindings/media/i2c/ov5640.txt b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
new file mode 100644
index 0000000..540b36c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
@@ -0,0 +1,45 @@
+* Omnivision OV5640 MIPI CSI-2 sensor
+
+Required Properties:
+- compatible: should be "ovti,ov5640"
+- clocks: reference to the xclk input clock.
+- clock-names: should be "xclk".
+- DOVDD-supply: Digital I/O voltage supply, 1.8 volts
+- AVDD-supply: Analog voltage supply, 2.8 volts
+- DVDD-supply: Digital core voltage supply, 1.5 volts
+
+Optional Properties:
+- reset-gpios: reference to the GPIO connected to the reset pin, if any.
+ This is an active low signal to the OV5640.
+- powerdown-gpios: reference to the GPIO connected to the powerdown pin,
+ if any. This is an active high signal to the OV5640.
+
+The device node must contain one 'port' child node for its digital output
+video port, in accordance with the video interface bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+Example:
+
+&i2c1 {
+ remote-endpoint = <&mipi_csi2_from_ov5640>;
.../devicetree/bindings/media/i2c/ov5640.txt | 45 ++++++++++++++++++++++
1 file changed, 45 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
diff --git a/Documentation/devicetree/bindings/media/i2c/ov5640.txt b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
new file mode 100644
index 0000000..540b36c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
@@ -0,0 +1,45 @@
+* Omnivision OV5640 MIPI CSI-2 sensor
+
+Required Properties:
+- compatible: should be "ovti,ov5640"
+- clocks: reference to the xclk input clock.
+- clock-names: should be "xclk".
+- DOVDD-supply: Digital I/O voltage supply, 1.8 volts
+- AVDD-supply: Analog voltage supply, 2.8 volts
+- DVDD-supply: Digital core voltage supply, 1.5 volts
+
+Optional Properties:
+- reset-gpios: reference to the GPIO connected to the reset pin, if any.
+ This is an active low signal to the OV5640.
+- powerdown-gpios: reference to the GPIO connected to the powerdown pin,
+ if any. This is an active high signal to the OV5640.
+
+The device node must contain one 'port' child node for its digital output
+video port, in accordance with the video interface bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+Example:
+
+&i2c1 {
+ ov5640: camera@3c {
+ compatible = "ovti,ov5640";
+ pinctrl-names = "default";
+ compatible = "ovti,ov5640";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ reg = <0x3c>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ AVDD-supply = <&vgen3_reg>; /* 2.8v */
+ reg = <0x3c>;
+ clocks = <&clks IMX6QDL_CLK_CKO>;
+ clock-names = "xclk";
+ DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+ DVDD-supply = <&vgen2_reg>; /* 1.5v */
+ powerdown-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>;
+
+ port {
+ ov5640_to_mipi_csi2: endpoint {
+ reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>;
+
+ port {
+ remote-endpoint = <&mipi_csi2_from_ov5640>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ data-lanes = <1 2>;
Steve Longerbeam
Mar 10, 2017, 12:00:07 AM3/10/17
to
Adds the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
Both hang off the same i2c2 bus, so they require different (and non-
default) i2c slave addresses.
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.
The OV5642 node is disabled temporarily while the subdev driver is
cleaned up and submitted later.
arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 ++
arch/arm/boot/dts/imx6q-sabrelite.dts | 5 ++
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 148 +++++++++++++++++++++++++++++++
3 files changed, 158 insertions(+)
diff --git a/arch/arm/boot/dts/imx6dl-sabrelite.dts b/arch/arm/boot/dts/imx6dl-sabrelite.dts
index 2f90452..3304076 100644
--- a/arch/arm/boot/dts/imx6dl-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6dl-sabrelite.dts
@@ -48,3 +48,8 @@
model = "Freescale i.MX6 DualLite SABRE Lite Board";
compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl";
};
+
+&ipu1_csi1_from_ipu1_csi1_mux {
index 02a7cdf..dc51262e 100644
--- a/arch/arm/boot/dts/imx6q-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6q-sabrelite.dts
@@ -52,3 +52,8 @@
index 89dce27..afe7449 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -39,6 +39,8 @@
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
+
pinctrl-0 = <&pinctrl_can_xcvr>;
gpio = <&gpio1 2 GPIO_ACTIVE_LOW>;
};
+
+ reg_1p5v: regulator@4 {
+ compatible = "regulator-fixed";
+ reg = <4>;
+ regulator-name = "1P5V";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <1500000>;
+ regulator-always-on;
+ };
+
+ reg_1p8v: regulator@5 {
+ compatible = "regulator-fixed";
+ reg = <5>;
+ regulator-name = "1P8V";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ };
+
+ reg_2p8v: regulator@6 {
+ compatible = "regulator-fixed";
+ reg = <6>;
+ regulator-name = "2P8V";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-always-on;
+ };
+ };
+
+ mipi_xclk: mipi_xclk {
+ compatible = "pwm-clock";
+ #clock-cells = <0>;
+ clock-frequency = <22000000>;
+ clock-output-names = "mipi_pwm3";
+ pwms = <&pwm3 0 45>; /* 1 / 45 ns = 22 MHz */
+ status = "okay";
};
gpio-keys {
@@ -220,6 +258,22 @@
+ pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
&audmux {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_audmux>;
@@ -298,6 +352,53 @@
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c2>;
status = "okay";
+
+ ov5640: camera@40 {
+ clocks = <&mipi_xclk>;
+ clock-names = "xclk";
+ DOVDD-supply = <®_1p8v>;
+ AVDD-supply = <®_2p8v>;
+ DVDD-supply = <®_1p5v>;
+ reset-gpios = <&gpio2 5 GPIO_ACTIVE_LOW>; /* NANDF_D5 */
+ powerdown-gpios = <&gpio6 9 GPIO_ACTIVE_HIGH>; /* NANDF_WP_B */
+
+ port {
+ remote-endpoint = <&mipi_csi2_in>;
+ compatible = "ovti,ov5642";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5642>;
+ clocks = <&clks IMX6QDL_CLK_CKO2>;
+ clock-names = "xclk";
+ reg = <0x42>;
+ reset-gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;
+ powerdown-gpios = <&gpio1 6 GPIO_ACTIVE_HIGH>;
+ gp-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
@@ -411,6 +512,23 @@
+ >;
+ };
+
pinctrl_j15: j15grp {
fsl,pins = <
MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10
@@ -444,6 +562,22 @@
>;
};
+ pinctrl_ov5640: ov5640grp {
+ fsl,pins = <
+ MX6QDL_PAD_NANDF_D5__GPIO2_IO05 0x000b0
+ MX6QDL_PAD_NANDF_WP_B__GPIO6_IO09 0x0b0b0
+ MX6QDL_PAD_GPIO_8__GPIO1_IO08 0x130b0
+ MX6QDL_PAD_GPIO_3__CCM_CLKO2 0x000b0
+ >;
+ };
+
pinctrl_pwm1: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD1_DAT3__PWM1_OUT 0x1b0b1
@@ -598,3 +732,17 @@
vmmc-supply = <®_3p3v>;
status = "okay";
};
+
+&mipi_csi {
2.7.4
Both hang off the same i2c2 bus, so they require different (and non-
default) i2c slave addresses.
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.
cleaned up and submitted later.
arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 ++
arch/arm/boot/dts/imx6q-sabrelite.dts | 5 ++
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 148 +++++++++++++++++++++++++++++++
3 files changed, 158 insertions(+)
diff --git a/arch/arm/boot/dts/imx6dl-sabrelite.dts b/arch/arm/boot/dts/imx6dl-sabrelite.dts
index 2f90452..3304076 100644
--- a/arch/arm/boot/dts/imx6dl-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6dl-sabrelite.dts
@@ -48,3 +48,8 @@
model = "Freescale i.MX6 DualLite SABRE Lite Board";
compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl";
};
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabrelite.dts b/arch/arm/boot/dts/imx6q-sabrelite.dts
+ data-lanes = <1 2>;
+};
index 02a7cdf..dc51262e 100644
--- a/arch/arm/boot/dts/imx6q-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6q-sabrelite.dts
@@ -52,3 +52,8 @@
&sata {
status = "okay";
};
+
+&ipu1_csi1_from_mipi_vc1 {
status = "okay";
};
+
+&ipu1_csi1_from_mipi_vc1 {
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+ data-lanes = <1 2>;
+};
index 89dce27..afe7449 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -39,6 +39,8 @@
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
+
+#include <dt-bindings/clock/imx6qdl-clock.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
@@ -94,6 +96,42 @@
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
pinctrl-0 = <&pinctrl_can_xcvr>;
gpio = <&gpio1 2 GPIO_ACTIVE_LOW>;
};
+
+ reg_1p5v: regulator@4 {
+ compatible = "regulator-fixed";
+ reg = <4>;
+ regulator-name = "1P5V";
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <1500000>;
+ regulator-always-on;
+ };
+
+ reg_1p8v: regulator@5 {
+ compatible = "regulator-fixed";
+ reg = <5>;
+ regulator-name = "1P8V";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-always-on;
+ };
+
+ reg_2p8v: regulator@6 {
+ compatible = "regulator-fixed";
+ reg = <6>;
+ regulator-name = "2P8V";
+ regulator-min-microvolt = <2800000>;
+ regulator-max-microvolt = <2800000>;
+ regulator-always-on;
+ };
+ };
+
+ mipi_xclk: mipi_xclk {
+ compatible = "pwm-clock";
+ #clock-cells = <0>;
+ clock-frequency = <22000000>;
+ clock-output-names = "mipi_pwm3";
+ pwms = <&pwm3 0 45>; /* 1 / 45 ns = 22 MHz */
+ status = "okay";
};
gpio-keys {
@@ -220,6 +258,22 @@
};
};
+&ipu1_csi0_from_ipu1_csi0_mux {
+ bus-width = <8>;
+ data-shift = <12>; /* Lines 19:12 used */
+ hsync-active = <1>;
+ vync-active = <1>;
};
+&ipu1_csi0_from_ipu1_csi0_mux {
+ bus-width = <8>;
+ data-shift = <12>; /* Lines 19:12 used */
+ hsync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+ remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+ pinctrl-names = "default";
+
+&ipu1_csi0_mux_from_parallel_sensor {
+ remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+ pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
&audmux {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_audmux>;
@@ -298,6 +352,53 @@
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_i2c2>;
status = "okay";
+
+ ov5640: camera@40 {
+ compatible = "ovti,ov5640";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ reg = <0x40>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5640>;
+ clocks = <&mipi_xclk>;
+ clock-names = "xclk";
+ DOVDD-supply = <®_1p8v>;
+ AVDD-supply = <®_2p8v>;
+ DVDD-supply = <®_1p5v>;
+ reset-gpios = <&gpio2 5 GPIO_ACTIVE_LOW>; /* NANDF_D5 */
+ powerdown-gpios = <&gpio6 9 GPIO_ACTIVE_HIGH>; /* NANDF_WP_B */
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ov5640_to_mipi_csi2: endpoint {
+ #size-cells = <0>;
+
+ remote-endpoint = <&mipi_csi2_in>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
+ ov5642: camera@42 {
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
+ compatible = "ovti,ov5642";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_ov5642>;
+ clocks = <&clks IMX6QDL_CLK_CKO2>;
+ clock-names = "xclk";
+ reg = <0x42>;
+ reset-gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;
+ powerdown-gpios = <&gpio1 6 GPIO_ACTIVE_HIGH>;
+ gp-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+ status = "disabled";
+
+ port {
+ ov5642_to_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+ bus-width = <8>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ };
+ };
+ };
};
&i2c3 {
+
+ port {
+ ov5642_to_ipu1_csi0_mux: endpoint {
+ remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+ bus-width = <8>;
+ hsync-active = <1>;
+ vsync-active = <1>;
+ };
+ };
+ };
};
@@ -411,6 +512,23 @@
>;
};
+ pinctrl_ipu1_csi0: ipu1csi0grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0
+ MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0
+ MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0
+ MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0
+ MX6QDL_PAD_CSI0_DATA_EN__IPU1_CSI0_DATA_EN 0x1b0b0
};
+ pinctrl_ipu1_csi0: ipu1csi0grp {
+ fsl,pins = <
+ MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0
+ MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0
+ MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0
+ MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0
+ MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0
+ >;
+ };
+
pinctrl_j15: j15grp {
fsl,pins = <
MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10
@@ -444,6 +562,22 @@
>;
};
+ pinctrl_ov5640: ov5640grp {
+ fsl,pins = <
+ MX6QDL_PAD_NANDF_WP_B__GPIO6_IO09 0x0b0b0
+ >;
+ };
+
+ pinctrl_ov5642: ov5642grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+ MX6QDL_PAD_GPIO_6__GPIO1_IO06 0x1b0b0
+ };
+
+ pinctrl_ov5642: ov5642grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x1b0b0
+ MX6QDL_PAD_GPIO_8__GPIO1_IO08 0x130b0
+ MX6QDL_PAD_GPIO_3__CCM_CLKO2 0x000b0
+ >;
+ };
+
pinctrl_pwm1: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD1_DAT3__PWM1_OUT 0x1b0b1
@@ -598,3 +732,17 @@
vmmc-supply = <®_3p3v>;
status = "okay";
};
+
+&mipi_csi {
+ status = "okay";
+
+ port@0 {
+ reg = <0>;
+
+
+ port@0 {
+ reg = <0>;
+
+ mipi_csi2_in: endpoint {
+ remote-endpoint = <&ov5640_to_mipi_csi2>;
+ remote-endpoint = <&ov5640_to_mipi_csi2>;
+ clock-lanes = <0>;
+ data-lanes = <1 2>;
+ };
+ };
+};
--
+ data-lanes = <1 2>;
+ };
+ };
+};
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:07 AM3/10/17
to
Add an empty UAPI Kbuild file for media UAPI headers.
include/uapi/Kbuild | 1 +
include/uapi/media/Kbuild | 1 +
2 files changed, 2 insertions(+)
create mode 100644 include/uapi/media/Kbuild
diff --git a/include/uapi/Kbuild b/include/uapi/Kbuild
index 245aa6e..9a51957 100644
--- a/include/uapi/Kbuild
+++ b/include/uapi/Kbuild
@@ -6,6 +6,7 @@
header-y += asm-generic/
header-y += linux/
header-y += sound/
+header-y += media/
header-y += mtd/
header-y += rdma/
header-y += video/
diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
new file mode 100644
index 0000000..aafaa5a
--- /dev/null
+++ b/include/uapi/media/Kbuild
@@ -0,0 +1 @@
+# UAPI Header export list
--
2.7.4
include/uapi/Kbuild | 1 +
include/uapi/media/Kbuild | 1 +
2 files changed, 2 insertions(+)
create mode 100644 include/uapi/media/Kbuild
diff --git a/include/uapi/Kbuild b/include/uapi/Kbuild
index 245aa6e..9a51957 100644
--- a/include/uapi/Kbuild
+++ b/include/uapi/Kbuild
@@ -6,6 +6,7 @@
header-y += asm-generic/
header-y += linux/
header-y += sound/
+header-y += media/
header-y += mtd/
header-y += rdma/
header-y += video/
diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
new file mode 100644
index 0000000..aafaa5a
--- /dev/null
+++ b/include/uapi/media/Kbuild
@@ -0,0 +1 @@
+# UAPI Header export list
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:07 AM3/10/17
to
From: Russell King <rmk+k...@armlinux.org.uk>
Add support to CSI for negotiation of frame intervals, and use this
information to configure the frame interval monitor.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
drivers/staging/media/imx/imx-media-csi.c | 46 +++++++++++++++++++++++++++++--
drivers/staging/media/imx/imx-media-fim.c | 28 +++++++------------
drivers/staging/media/imx/imx-media.h | 2 +-
3 files changed, 54 insertions(+), 22 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 6640869..b556fa4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -63,6 +63,7 @@ struct csi_priv {
struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
struct v4l2_rect crop;
/* active vb2 buffers to send to video dev sink */
@@ -570,7 +571,8 @@ static int csi_start(struct csi_priv *priv)
/* start the frame interval monitor */
if (priv->fim) {
- ret = imx_media_fim_set_stream(priv->fim, priv->sensor, true);
+ ret = imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, true);
if (ret)
goto idmac_stop;
}
@@ -585,7 +587,8 @@ static int csi_start(struct csi_priv *priv)
fim_off:
if (priv->fim)
- imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+ imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, false);
idmac_stop:
@@ -599,7 +602,8 @@ static void csi_stop(struct csi_priv *priv)
/* stop the frame interval monitor */
if (priv->fim)
- imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+ imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, false);
ipu_csi_disable(priv->csi);
}
@@ -608,6 +612,36 @@ static void csi_stop(struct csi_priv *priv)
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ mutex_unlock(&priv->lock);
goto put_csi;
ret = PTR_ERR(priv->fim);
@@ -1254,6 +1292,8 @@ static struct v4l2_subdev_core_ops csi_core_ops = {
};
static struct v4l2_subdev_video_ops csi_video_ops = {
+ .g_frame_interval = csi_g_frame_interval,
+ .s_frame_interval = csi_s_frame_interval,
.s_stream = csi_s_stream,
};
diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
index 824d257..4fe5ffc 100644
--- a/drivers/staging/media/imx/imx-media-fim.c
+++ b/drivers/staging/media/imx/imx-media-fim.c
@@ -67,26 +67,18 @@ struct imx_media_fim {
};
static void update_fim_nominal(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor)
+ const struct v4l2_fract *fi)
{
- struct v4l2_streamparm parm;
- struct v4l2_fract tpf;
- int ret;
-
- parm.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
- ret = v4l2_subdev_call(sensor->sd, video, g_parm, &parm);
- tpf = parm.parm.capture.timeperframe;
-
- if (ret || tpf.denominator == 0) {
- dev_dbg(fim->sd->dev, "no tpf from sensor, FIM disabled\n");
+ if (fi->denominator == 0) {
+ dev_dbg(fim->sd->dev, "no frame interval, FIM disabled\n");
fim->enabled = false;
return;
}
- fim->nominal = DIV_ROUND_CLOSEST(1000 * 1000 * tpf.numerator,
- tpf.denominator);
+ fim->nominal = DIV_ROUND_CLOSEST_ULL(1000000ULL * (u64)fi->numerator,
+ fi->denominator);
- dev_dbg(fim->sd->dev, "sensor FI=%lu usec\n", fim->nominal);
+ dev_dbg(fim->sd->dev, "FI=%lu usec\n", fim->nominal);
}
static void reset_fim(struct imx_media_fim *fim, bool curval)
@@ -130,8 +122,8 @@ static void send_fim_event(struct imx_media_fim *fim, unsigned long error)
/*
* Monitor an averaged frame interval. If the average deviates too much
- * from the sensor's nominal frame rate, send the frame interval error
- * event. The frame intervals are averaged in order to quiet noise from
+ * from the nominal frame rate, send the frame interval error event. The
+ * frame intervals are averaged in order to quiet noise from
* (presumably random) interrupt latency.
*/
static void frame_interval_monitor(struct imx_media_fim *fim,
@@ -422,12 +414,12 @@ EXPORT_SYMBOL_GPL(imx_media_fim_set_power);
/* Called by the subdev in its s_stream callback */
int imx_media_fim_set_stream(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor,
+ const struct v4l2_fract *fi,
bool on)
{
if (on) {
reset_fim(fim, true);
- update_fim_nominal(fim, sensor);
+ update_fim_nominal(fim, fi);
if (fim->icap_channel >= 0)
fim_acquire_first_ts(fim);
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 9e865fa..a67ee14 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -262,7 +262,7 @@ struct imx_media_fim;
void imx_media_fim_eof_monitor(struct imx_media_fim *fim, struct timespec *ts);
int imx_media_fim_set_power(struct imx_media_fim *fim, bool on);
int imx_media_fim_set_stream(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor,
+ const struct v4l2_fract *frame_interval,
bool on);
struct imx_media_fim *imx_media_fim_init(struct v4l2_subdev *sd);
void imx_media_fim_free(struct imx_media_fim *fim);
--
2.7.4
Add support to CSI for negotiation of frame intervals, and use this
information to configure the frame interval monitor.
Signed-off-by: Russell King <rmk+k...@armlinux.org.uk>
drivers/staging/media/imx/imx-media-csi.c | 46 +++++++++++++++++++++++++++++--
drivers/staging/media/imx/imx-media-fim.c | 28 +++++++------------
drivers/staging/media/imx/imx-media.h | 2 +-
3 files changed, 54 insertions(+), 22 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 6640869..b556fa4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -63,6 +63,7 @@ struct csi_priv {
struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
+ struct v4l2_fract frame_interval;
struct v4l2_rect crop;
/* active vb2 buffers to send to video dev sink */
/* start the frame interval monitor */
- ret = imx_media_fim_set_stream(priv->fim, priv->sensor, true);
+ ret = imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, true);
if (ret)
goto idmac_stop;
}
@@ -585,7 +587,8 @@ static int csi_start(struct csi_priv *priv)
fim_off:
if (priv->fim)
- imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+ imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, false);
idmac_stop:
if (priv->dest == IPU_CSI_DEST_IDMAC)
csi_idmac_stop(priv);
@@ -599,7 +602,8 @@ static void csi_stop(struct csi_priv *priv)
/* stop the frame interval monitor */
- imx_media_fim_set_stream(priv->fim, priv->sensor, false);
+ imx_media_fim_set_stream(priv->fim,
+ &priv->frame_interval, false);
ipu_csi_disable(priv->csi);
}
@@ -608,6 +612,36 @@ static void csi_stop(struct csi_priv *priv)
* V4L2 subdev operations.
*/
+static int csi_g_frame_interval(struct v4l2_subdev *sd,
*/
+ struct v4l2_subdev_frame_interval *fi)
+{
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+ mutex_lock(&priv->lock);
+
+ fi->interval = priv->frame_interval;
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+static int csi_s_frame_interval(struct v4l2_subdev *sd,
+ return 0;
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+{
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+
+ mutex_lock(&priv->lock);
+
+
+ /* Output pads mirror active input pad, no limits on input pads */
+ if (fi->pad == CSI_SRC_PAD_IDMAC || fi->pad == CSI_SRC_PAD_DIRECT)
+ /* Output pads mirror active input pad, no limits on input pads */
+ fi->interval = priv->frame_interval;
+
+ priv->frame_interval = fi->interval;
+
+
+ mutex_unlock(&priv->lock);
+
+ return 0;
+}
+
+ return 0;
+}
+
static int csi_s_stream(struct v4l2_subdev *sd, int enable)
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
@@ -1204,6 +1238,10 @@ static int csi_registered(struct v4l2_subdev *sd)
goto put_csi;
}
+ /* init default frame interval */
+ /* init default frame interval */
+ priv->frame_interval.numerator = 1;
+ priv->frame_interval.denominator = 30;
+ priv->frame_interval.denominator = 30;
+
priv->fim = imx_media_fim_init(&priv->sd);
if (IS_ERR(priv->fim)) {
priv->fim = imx_media_fim_init(&priv->sd);
ret = PTR_ERR(priv->fim);
@@ -1254,6 +1292,8 @@ static struct v4l2_subdev_core_ops csi_core_ops = {
};
static struct v4l2_subdev_video_ops csi_video_ops = {
+ .s_frame_interval = csi_s_frame_interval,
.s_stream = csi_s_stream,
};
diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
index 824d257..4fe5ffc 100644
--- a/drivers/staging/media/imx/imx-media-fim.c
+++ b/drivers/staging/media/imx/imx-media-fim.c
@@ -67,26 +67,18 @@ struct imx_media_fim {
};
static void update_fim_nominal(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor)
+ const struct v4l2_fract *fi)
{
- struct v4l2_streamparm parm;
- struct v4l2_fract tpf;
- int ret;
-
- parm.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
- ret = v4l2_subdev_call(sensor->sd, video, g_parm, &parm);
- tpf = parm.parm.capture.timeperframe;
-
- if (ret || tpf.denominator == 0) {
- dev_dbg(fim->sd->dev, "no tpf from sensor, FIM disabled\n");
+ if (fi->denominator == 0) {
+ dev_dbg(fim->sd->dev, "no frame interval, FIM disabled\n");
fim->enabled = false;
return;
}
- fim->nominal = DIV_ROUND_CLOSEST(1000 * 1000 * tpf.numerator,
- tpf.denominator);
+ fim->nominal = DIV_ROUND_CLOSEST_ULL(1000000ULL * (u64)fi->numerator,
+ fi->denominator);
- dev_dbg(fim->sd->dev, "sensor FI=%lu usec\n", fim->nominal);
+ dev_dbg(fim->sd->dev, "FI=%lu usec\n", fim->nominal);
}
static void reset_fim(struct imx_media_fim *fim, bool curval)
@@ -130,8 +122,8 @@ static void send_fim_event(struct imx_media_fim *fim, unsigned long error)
/*
* Monitor an averaged frame interval. If the average deviates too much
- * from the sensor's nominal frame rate, send the frame interval error
- * event. The frame intervals are averaged in order to quiet noise from
+ * from the nominal frame rate, send the frame interval error event. The
+ * frame intervals are averaged in order to quiet noise from
* (presumably random) interrupt latency.
*/
static void frame_interval_monitor(struct imx_media_fim *fim,
@@ -422,12 +414,12 @@ EXPORT_SYMBOL_GPL(imx_media_fim_set_power);
/* Called by the subdev in its s_stream callback */
int imx_media_fim_set_stream(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor,
+ const struct v4l2_fract *fi,
bool on)
{
if (on) {
reset_fim(fim, true);
- update_fim_nominal(fim, sensor);
+ update_fim_nominal(fim, fi);
if (fim->icap_channel >= 0)
fim_acquire_first_ts(fim);
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 9e865fa..a67ee14 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -262,7 +262,7 @@ struct imx_media_fim;
void imx_media_fim_eof_monitor(struct imx_media_fim *fim, struct timespec *ts);
int imx_media_fim_set_power(struct imx_media_fim *fim, bool on);
int imx_media_fim_set_stream(struct imx_media_fim *fim,
- struct imx_media_subdev *sensor,
+ const struct v4l2_fract *frame_interval,
bool on);
struct imx_media_fim *imx_media_fim_init(struct v4l2_subdev *sd);
void imx_media_fim_free(struct imx_media_fim *fim);
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:00:08 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
The CSI can skip any out of up to 6 input frames, allowing to reduce the
frame rate at the output pads by small fractions.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
drivers/staging/media/imx/imx-media-csi.c | 125 ++++++++++++++++++++++++++++--
1 file changed, 120 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index a726744..e5105ec 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1,13 +1,15 @@
/*
- * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ * Copyright (c) 2014-2017 Mentor Graphics Inc.
+ * Copyright (C) 2017 Pengutronix, Philipp Zabel <ker...@pengutronix.de>
*
+#include <linux/gcd.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
@@ -40,6 +42,18 @@
+ * struct csi_skip_desc - CSI frame skipping descriptor
+ * @keep - number of frames kept per max_ratio frames
+ * @max_ratio - width of skip_smfc, written to MAX_RATIO bitfield
+ * @skip_smfc - skip pattern written to the SKIP_SMFC bitfield
+ */
+struct csi_skip_desc {
+ u8 keep;
+ u8 max_ratio;
+ u8 skip_smfc;
+};
+
struct csi_priv {
struct device *dev;
struct ipu_soc *ipu;
@@ -65,6 +79,7 @@ struct csi_priv {
/* active vb2 buffers to send to video dev sink */
struct imx_media_buffer *active_vb2_buf[2];
@@ -517,10 +532,12 @@ static int csi_setup(struct csi_priv *priv)
struct v4l2_mbus_config sensor_mbus_cfg;
struct v4l2_of_endpoint *sensor_ep;
struct v4l2_mbus_framefmt if_fmt;
+ const struct csi_skip_desc *skip;
infmt = &priv->format_mbus[CSI_SINK_PAD];
/* compose mbus_config from sensor endpoint */
sensor_mbus_cfg.type = sensor_ep->bus_type;
@@ -545,6 +562,9 @@ static int csi_setup(struct csi_priv *priv)
ipu_csi_set_dest(priv->csi, priv->dest);
+ ipu_csi_set_skip_smfc(priv->csi, skip->skip_smfc, skip->max_ratio - 1,
+ 0);
+
ipu_csi_dump(priv->csi);
return 0;
@@ -608,6 +628,77 @@ static void csi_stop(struct csi_priv *priv)
ipu_csi_disable(priv->csi);
}
+static const struct csi_skip_desc csi_skip[12] = {
+ { 1, 1, 0x00 }, /* Keep all frames */
+ { 5, 6, 0x10 }, /* Skip every sixth frame */
+ { 4, 5, 0x08 }, /* Skip every fifth frame */
+ { 3, 4, 0x04 }, /* Skip every fourth frame */
+ { 2, 3, 0x02 }, /* Skip every third frame */
+ { 3, 5, 0x0a }, /* Skip frames 1 and 3 of every 5 */
+ { 1, 2, 0x01 }, /* Skip every second frame */
+ { 2, 5, 0x0b }, /* Keep frames 1 and 4 of every 5 */
+ { 1, 3, 0x03 }, /* Keep one in three frames */
+ { 1, 4, 0x07 }, /* Keep one in four frames */
+ { 1, 5, 0x0f }, /* Keep one in five frames */
+ { 1, 6, 0x1f }, /* Keep one in six frames */
+};
+
+static void csi_apply_skip_interval(const struct csi_skip_desc *skip,
+ struct v4l2_fract *interval)
+{
+ unsigned int div;
+
+ interval->numerator *= skip->max_ratio;
+ interval->denominator *= skip->keep;
+
+ /* Reduce fraction to lowest terms */
+ div = gcd(interval->numerator, interval->denominator);
+ if (div > 1) {
+ interval->numerator /= div;
+ interval->denominator /= div;
+ }
+}
+
+/*
+ * Find the skip pattern to produce the output frame interval closest to the
+ * requested one, for the given input frame interval. Updates the output frame
+ * interval to the exact value.
+ */
+static const struct csi_skip_desc *csi_find_best_skip(struct v4l2_fract *in,
+ struct v4l2_fract *out)
+{
+ const struct csi_skip_desc *skip = &csi_skip[0], *best_skip = skip;
+ u32 min_err = UINT_MAX;
+ u64 want_us;
+ int i;
+
+ /* Default to 1:1 ratio */
+ if (out->numerator == 0 || out->denominator == 0 ||
+ in->numerator == 0 || in->denominator == 0)
+ return best_skip;
+
+ want_us = div_u64((u64)USEC_PER_SEC * out->numerator, out->denominator);
+
+ /* Find the reduction closest to the requested time per frame */
+ for (i = 0; i < ARRAY_SIZE(csi_skip); i++, skip++) {
+ u64 tmp, err;
+
+ tmp = div_u64((u64)USEC_PER_SEC * in->numerator *
+ skip->max_ratio, in->denominator * skip->keep);
+
+ err = abs((s64)tmp - want_us);
+ if (err < min_err) {
+ min_err = err;
+ best_skip = skip;
+ }
+ }
+
+ *out = *in;
+ csi_apply_skip_interval(best_skip, out);
+
+ return best_skip;
+}
+
/*
{
+ return -EINVAL;
+ csi_apply_skip_interval(priv->skip[fi->pad - 1], &fi->interval);
+
mutex_unlock(&priv->lock);
return 0;
@@ -629,14 +728,27 @@ static int csi_s_frame_interval(struct v4l2_subdev *sd,
{
+ return -EINVAL;
+
mutex_lock(&priv->lock);
- /* Output pads mirror active input pad, no limits on input pads */
- if (fi->pad == CSI_SRC_PAD_IDMAC || fi->pad == CSI_SRC_PAD_DIRECT)
- fi->interval = priv->frame_interval;
+ /* No limits on input pad */
+ if (fi->pad == CSI_SINK_PAD) {
+ priv->skip[0] = &csi_skip[0];
+ priv->skip[1] = &csi_skip[0];
- priv->frame_interval = fi->interval;
+ goto out;
+ }
+
+ /* Output pads depend on input interval, modified by frame skipping */
+ priv->skip[fi->pad - 1] = csi_find_best_skip(&priv->frame_interval,
+ &fi->interval);
+out:
mutex_unlock(&priv->lock);
return 0;
@@ -1256,6 +1368,9 @@ static int csi_registered(struct v4l2_subdev *sd)
&priv->cc[i]);
if (ret)
goto put_csi;
+ /* disable frame skipping */
2.7.4
The CSI can skip any out of up to 6 input frames, allowing to reduce the
frame rate at the output pads by small fractions.
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
drivers/staging/media/imx/imx-media-csi.c | 125 ++++++++++++++++++++++++++++--
1 file changed, 120 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index a726744..e5105ec 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1,13 +1,15 @@
/*
* V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
*
- * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ * Copyright (c) 2014-2017 Mentor Graphics Inc.
+ * Copyright (C) 2017 Pengutronix, Philipp Zabel <ker...@pengutronix.de>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*/
+#include <linux/gcd.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
@@ -40,6 +42,18 @@
#define H_ALIGN 1 /* multiple of 2 lines */
#define S_ALIGN 1 /* multiple of 2 */
+/*
+ * struct csi_skip_desc - CSI frame skipping descriptor
+ * @keep - number of frames kept per max_ratio frames
+ * @max_ratio - width of skip_smfc, written to MAX_RATIO bitfield
+ * @skip_smfc - skip pattern written to the SKIP_SMFC bitfield
+ */
+struct csi_skip_desc {
+ u8 keep;
+ u8 max_ratio;
+ u8 skip_smfc;
+};
+
struct csi_priv {
struct device *dev;
struct ipu_soc *ipu;
@@ -65,6 +79,7 @@ struct csi_priv {
const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
struct v4l2_fract frame_interval;
struct v4l2_rect crop;
+ const struct csi_skip_desc *skip[CSI_NUM_PADS - 1];
struct v4l2_rect crop;
/* active vb2 buffers to send to video dev sink */
@@ -517,10 +532,12 @@ static int csi_setup(struct csi_priv *priv)
struct v4l2_mbus_config sensor_mbus_cfg;
struct v4l2_of_endpoint *sensor_ep;
struct v4l2_mbus_framefmt if_fmt;
+ const struct csi_skip_desc *skip;
infmt = &priv->format_mbus[CSI_SINK_PAD];
outfmt = &priv->format_mbus[priv->active_output_pad];
sensor_ep = &priv->sensor->sensor_ep;
+ skip = priv->skip[priv->active_output_pad - 1];
/* compose mbus_config from sensor endpoint */
@@ -545,6 +562,9 @@ static int csi_setup(struct csi_priv *priv)
ipu_csi_set_dest(priv->csi, priv->dest);
+ ipu_csi_set_skip_smfc(priv->csi, skip->skip_smfc, skip->max_ratio - 1,
+ 0);
+
ipu_csi_dump(priv->csi);
return 0;
@@ -608,6 +628,77 @@ static void csi_stop(struct csi_priv *priv)
ipu_csi_disable(priv->csi);
}
+static const struct csi_skip_desc csi_skip[12] = {
+ { 1, 1, 0x00 }, /* Keep all frames */
+ { 5, 6, 0x10 }, /* Skip every sixth frame */
+ { 4, 5, 0x08 }, /* Skip every fifth frame */
+ { 3, 4, 0x04 }, /* Skip every fourth frame */
+ { 2, 3, 0x02 }, /* Skip every third frame */
+ { 3, 5, 0x0a }, /* Skip frames 1 and 3 of every 5 */
+ { 1, 2, 0x01 }, /* Skip every second frame */
+ { 2, 5, 0x0b }, /* Keep frames 1 and 4 of every 5 */
+ { 1, 3, 0x03 }, /* Keep one in three frames */
+ { 1, 4, 0x07 }, /* Keep one in four frames */
+ { 1, 5, 0x0f }, /* Keep one in five frames */
+ { 1, 6, 0x1f }, /* Keep one in six frames */
+};
+
+static void csi_apply_skip_interval(const struct csi_skip_desc *skip,
+ struct v4l2_fract *interval)
+{
+ unsigned int div;
+
+ interval->numerator *= skip->max_ratio;
+ interval->denominator *= skip->keep;
+
+ /* Reduce fraction to lowest terms */
+ div = gcd(interval->numerator, interval->denominator);
+ if (div > 1) {
+ interval->numerator /= div;
+ interval->denominator /= div;
+ }
+}
+
+/*
+ * Find the skip pattern to produce the output frame interval closest to the
+ * requested one, for the given input frame interval. Updates the output frame
+ * interval to the exact value.
+ */
+static const struct csi_skip_desc *csi_find_best_skip(struct v4l2_fract *in,
+ struct v4l2_fract *out)
+{
+ const struct csi_skip_desc *skip = &csi_skip[0], *best_skip = skip;
+ u32 min_err = UINT_MAX;
+ u64 want_us;
+ int i;
+
+ /* Default to 1:1 ratio */
+ if (out->numerator == 0 || out->denominator == 0 ||
+ in->numerator == 0 || in->denominator == 0)
+ return best_skip;
+
+ want_us = div_u64((u64)USEC_PER_SEC * out->numerator, out->denominator);
+
+ /* Find the reduction closest to the requested time per frame */
+ for (i = 0; i < ARRAY_SIZE(csi_skip); i++, skip++) {
+ u64 tmp, err;
+
+ tmp = div_u64((u64)USEC_PER_SEC * in->numerator *
+ skip->max_ratio, in->denominator * skip->keep);
+
+ err = abs((s64)tmp - want_us);
+ if (err < min_err) {
+ min_err = err;
+ best_skip = skip;
+ }
+ }
+
+ *out = *in;
+ csi_apply_skip_interval(best_skip, out);
+
+ return best_skip;
+}
+
/*
* V4L2 subdev operations.
*/
@@ -617,8 +708,16 @@ static int csi_g_frame_interval(struct v4l2_subdev *sd,
*/
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ if (fi->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
mutex_lock(&priv->lock);
+
fi->interval = priv->frame_interval;
+
+ if (fi->pad != CSI_SINK_PAD)
mutex_lock(&priv->lock);
+
fi->interval = priv->frame_interval;
+
+ csi_apply_skip_interval(priv->skip[fi->pad - 1], &fi->interval);
+
mutex_unlock(&priv->lock);
return 0;
@@ -629,14 +728,27 @@ static int csi_s_frame_interval(struct v4l2_subdev *sd,
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ if (fi->pad >= CSI_NUM_PADS)
+ return -EINVAL;
+
mutex_lock(&priv->lock);
- /* Output pads mirror active input pad, no limits on input pads */
- if (fi->pad == CSI_SRC_PAD_IDMAC || fi->pad == CSI_SRC_PAD_DIRECT)
- fi->interval = priv->frame_interval;
+ /* No limits on input pad */
+ if (fi->pad == CSI_SINK_PAD) {
+ priv->frame_interval = fi->interval;
+
+ /* Reset frame skipping ratio to 1:1 */
+
+ priv->skip[0] = &csi_skip[0];
+ priv->skip[1] = &csi_skip[0];
- priv->frame_interval = fi->interval;
+ goto out;
+ }
+
+ /* Output pads depend on input interval, modified by frame skipping */
+ priv->skip[fi->pad - 1] = csi_find_best_skip(&priv->frame_interval,
+ &fi->interval);
+out:
mutex_unlock(&priv->lock);
return 0;
@@ -1256,6 +1368,9 @@ static int csi_registered(struct v4l2_subdev *sd)
&priv->cc[i]);
if (ret)
goto put_csi;
+ /* disable frame skipping */
+ if (i != CSI_SINK_PAD)
+ priv->skip[i - 1] = &csi_skip[0];
}
/* init default frame interval */
--
/* init default frame interval */
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:04 AM3/10/17
to
From: Philipp Zabel <p.z...@pengutronix.de>
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
- renamed MEDIA_ENT_F_MUX to MEDIA_ENT_F_VID_MUX
Documentation/media/uapi/mediactl/media-types.rst | 22 ++++++++++++++++++++++
include/uapi/linux/media.h | 6 ++++++
2 files changed, 28 insertions(+)
diff --git a/Documentation/media/uapi/mediactl/media-types.rst b/Documentation/media/uapi/mediactl/media-types.rst
index 3e03dc2..9d908fe 100644
--- a/Documentation/media/uapi/mediactl/media-types.rst
+++ b/Documentation/media/uapi/mediactl/media-types.rst
@@ -298,6 +298,28 @@ Types and flags used to represent the media graph elements
received on its sink pad and outputs the statistics data on
its source pad.
+ - .. row 29
+
+ .. _MEDIA-ENT-F-VID-MUX:
+
+ - ``MEDIA_ENT_F_VID_MUX``
+
+ - Video multiplexer. An entity capable of multiplexing must have at
+ least two sink pads and one source pad, and must pass the video
+ frame(s) received from the active sink pad to the source pad. Video
+ frame(s) from the inactive sink pads are discarded.
+
+ - .. row 30
+
+ .. _MEDIA-ENT-F-VID-IF-BRIDGE:
+
+ - ``MEDIA_ENT_F_VID_IF_BRIDGE``
+
+ - Video interface bridge. A video interface bridge entity must have at
+ least one sink pad and one source pad. It receives video frame(s) on
+ its sink pad in one bus format (HDMI, eDP, MIPI CSI-2, ...) and
+ converts them and outputs them on its source pad in another bus format
+ (eDP, MIPI CSI-2, parallel, ...).
.. tabularcolumns:: |p{5.5cm}|p{12.0cm}|
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 4890787..fac96c6 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -105,6 +105,12 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_STATISTICS (MEDIA_ENT_F_BASE + 0x4006)
/*
+ * Switch and bridge entitites
+ */
+#define MEDIA_ENT_F_VID_MUX (MEDIA_ENT_F_BASE + 0x5001)
+#define MEDIA_ENT_F_VID_IF_BRIDGE (MEDIA_ENT_F_BASE + 0x5002)
+
+/*
* Connectors
*/
/* It is a responsibility of the entity drivers to add connectors and links */
--
2.7.4
Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
- renamed MEDIA_ENT_F_MUX to MEDIA_ENT_F_VID_MUX
Documentation/media/uapi/mediactl/media-types.rst | 22 ++++++++++++++++++++++
include/uapi/linux/media.h | 6 ++++++
2 files changed, 28 insertions(+)
diff --git a/Documentation/media/uapi/mediactl/media-types.rst b/Documentation/media/uapi/mediactl/media-types.rst
index 3e03dc2..9d908fe 100644
--- a/Documentation/media/uapi/mediactl/media-types.rst
+++ b/Documentation/media/uapi/mediactl/media-types.rst
@@ -298,6 +298,28 @@ Types and flags used to represent the media graph elements
received on its sink pad and outputs the statistics data on
its source pad.
+ - .. row 29
+
+ .. _MEDIA-ENT-F-VID-MUX:
+
+ - ``MEDIA_ENT_F_VID_MUX``
+
+ - Video multiplexer. An entity capable of multiplexing must have at
+ least two sink pads and one source pad, and must pass the video
+ frame(s) received from the active sink pad to the source pad. Video
+ frame(s) from the inactive sink pads are discarded.
+
+ - .. row 30
+
+ .. _MEDIA-ENT-F-VID-IF-BRIDGE:
+
+ - ``MEDIA_ENT_F_VID_IF_BRIDGE``
+
+ - Video interface bridge. A video interface bridge entity must have at
+ least one sink pad and one source pad. It receives video frame(s) on
+ its sink pad in one bus format (HDMI, eDP, MIPI CSI-2, ...) and
+ converts them and outputs them on its source pad in another bus format
+ (eDP, MIPI CSI-2, parallel, ...).
.. tabularcolumns:: |p{5.5cm}|p{12.0cm}|
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 4890787..fac96c6 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -105,6 +105,12 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_STATISTICS (MEDIA_ENT_F_BASE + 0x4006)
/*
+ * Switch and bridge entitites
+ */
+#define MEDIA_ENT_F_VID_MUX (MEDIA_ENT_F_BASE + 0x5001)
+#define MEDIA_ENT_F_VID_IF_BRIDGE (MEDIA_ENT_F_BASE + 0x5002)
+
+/*
* Connectors
*/
/* It is a responsibility of the entity drivers to add connectors and links */
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:04 AM3/10/17
to
There is a pin conflict with GPIO_6. This pin functions as a power
input pin to the OV5642 camera sensor, but ENET uses it as the h/w
workaround for erratum ERR006687, to wake-up the ARM cores on normal
RX and TX packet done events. So we need to remove the h/w workaround
to support the OV5642. The result is that the CPUidle driver will no
longer allow entering the deep idle states on the sabrelite.
This is a partial revert of
commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
interrupt.")
commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
for ERR006687")
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 8413179..89dce27 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -270,9 +270,6 @@
txd1-skew-ps = <0>;
txd2-skew-ps = <0>;
txd3-skew-ps = <0>;
- interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>,
- <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
- fsl,err006687-workaround-present;
status = "okay";
};
@@ -373,7 +370,6 @@
MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL 0x1b030
/* Phy reset */
MX6QDL_PAD_EIM_D23__GPIO3_IO23 0x000b0
- MX6QDL_PAD_GPIO_6__ENET_IRQ 0x000b1
>;
};
--
2.7.4
input pin to the OV5642 camera sensor, but ENET uses it as the h/w
workaround for erratum ERR006687, to wake-up the ARM cores on normal
RX and TX packet done events. So we need to remove the h/w workaround
to support the OV5642. The result is that the CPUidle driver will no
longer allow entering the deep idle states on the sabrelite.
This is a partial revert of
commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
interrupt.")
commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
for ERR006687")
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 8413179..89dce27 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -270,9 +270,6 @@
txd1-skew-ps = <0>;
txd2-skew-ps = <0>;
txd3-skew-ps = <0>;
- interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>,
- <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
- fsl,err006687-workaround-present;
status = "okay";
};
@@ -373,7 +370,6 @@
MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL 0x1b030
/* Phy reset */
MX6QDL_PAD_EIM_D23__GPIO3_IO23 0x000b0
- MX6QDL_PAD_GPIO_6__ENET_IRQ 0x000b1
>;
};
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:04 AM3/10/17
to
arch/arm/boot/dts/imx6dl.dtsi | 5 +++++
arch/arm/boot/dts/imx6q.dtsi | 5 +++++
2 files changed, 10 insertions(+)
diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 8958c4a..a959c76 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -100,6 +100,11 @@
};
};
+ capture-subsystem {
+ compatible = "fsl,imx-capture-subsystem";
+ ports = <&ipu1_csi0>, <&ipu1_csi1>;
+ };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <&ipu1_di0>, <&ipu1_di1>;
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index b833b0d..4cc6579 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -206,6 +206,11 @@
};
};
+ capture-subsystem {
+ compatible = "fsl,imx-capture-subsystem";
+ ports = <&ipu1_csi0>, <&ipu1_csi1>, <&ipu2_csi0>, <&ipu2_csi1>;
+ };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <&ipu1_di0>, <&ipu1_di1>, <&ipu2_di0>, <&ipu2_di1>;
--
2.7.4
arch/arm/boot/dts/imx6q.dtsi | 5 +++++
2 files changed, 10 insertions(+)
diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 8958c4a..a959c76 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -100,6 +100,11 @@
};
};
+ capture-subsystem {
+ compatible = "fsl,imx-capture-subsystem";
+ ports = <&ipu1_csi0>, <&ipu1_csi1>;
+ };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <&ipu1_di0>, <&ipu1_di1>;
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index b833b0d..4cc6579 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -206,6 +206,11 @@
};
};
+ capture-subsystem {
+ compatible = "fsl,imx-capture-subsystem";
+ ports = <&ipu1_csi0>, <&ipu1_csi1>, <&ipu2_csi0>, <&ipu2_csi1>;
+ };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <&ipu1_di0>, <&ipu1_di1>, <&ipu2_di0>, <&ipu2_di1>;
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:04 AM3/10/17
to
Add to the MIPI CSI2 receiver node: compatible strings,
interrupt sources, and clocks.
arch/arm/boot/dts/imx6qdl.dtsi | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 6d7bf64..d28a413 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1134,7 +1134,14 @@
};
mipi_csi: mipi@021dc000 {
+ compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
reg = <0x021dc000 0x4000>;
+ interrupts = <0 100 0x04>, <0 101 0x04>;
+ clocks = <&clks IMX6QDL_CLK_HSI_TX>,
+ <&clks IMX6QDL_CLK_VIDEO_27M>,
+ <&clks IMX6QDL_CLK_EIM_PODF>;
+ clock-names = "dphy", "ref", "pix";
+ status = "disabled";
};
mipi_dsi: mipi@021e0000 {
--
2.7.4
interrupt sources, and clocks.
arch/arm/boot/dts/imx6qdl.dtsi | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 6d7bf64..d28a413 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1134,7 +1134,14 @@
};
mipi_csi: mipi@021dc000 {
+ compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
reg = <0x021dc000 0x4000>;
+ interrupts = <0 100 0x04>, <0 101 0x04>;
+ clocks = <&clks IMX6QDL_CLK_HSI_TX>,
+ <&clks IMX6QDL_CLK_VIDEO_27M>,
+ <&clks IMX6QDL_CLK_EIM_PODF>;
+ clock-names = "dphy", "ref", "pix";
+ status = "disabled";
};
mipi_dsi: mipi@021e0000 {
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:04 AM3/10/17
to
v4l2_pipeline_inherit_controls() will add the v4l2 controls from
all subdev entities in a pipeline to a given video device.
drivers/media/v4l2-core/v4l2-mc.c | 48 +++++++++++++++++++++++++++++++++++++++
include/media/v4l2-mc.h | 25 ++++++++++++++++++++
2 files changed, 73 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 303980b..09d4d97 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -22,6 +22,7 @@
#include <linux/usb.h>
#include <media/media-device.h>
#include <media/media-entity.h>
+#include <media/v4l2-ctrls.h>
#include <media/v4l2-fh.h>
#include <media/v4l2-mc.h>
#include <media/v4l2-subdev.h>
@@ -238,6 +239,53 @@ int v4l_vb2q_enable_media_source(struct vb2_queue *q)
}
EXPORT_SYMBOL_GPL(v4l_vb2q_enable_media_source);
+int __v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ struct media_device *mdev = start_entity->graph_obj.mdev;
+ struct media_entity *entity;
+ struct media_graph graph;
+ struct v4l2_subdev *sd;
+ int ret;
+
+ ret = media_graph_walk_init(&graph, mdev);
+
+ while ((entity = media_graph_walk_next(&graph))) {
+ if (!is_media_entity_v4l2_subdev(entity))
+ continue;
+
+ sd = media_entity_to_v4l2_subdev(entity);
+
+ ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
+ sd->ctrl_handler,
+ NULL);
+ if (ret)
+ break;
+ }
+
+ media_graph_walk_cleanup(&graph);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(__v4l2_pipeline_inherit_controls);
+
+int v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ struct media_device *mdev = start_entity->graph_obj.mdev;
+ int ret;
+
+ mutex_lock(&mdev->graph_mutex);
+ ret = __v4l2_pipeline_inherit_controls(vfd, start_entity);
+ mutex_unlock(&mdev->graph_mutex);
+
/* -----------------------------------------------------------------------------
* Pipeline power management
*
diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
index 2634d9d..9848e77 100644
--- a/include/media/v4l2-mc.h
+++ b/include/media/v4l2-mc.h
@@ -171,6 +171,17 @@ void v4l_disable_media_source(struct video_device *vdev);
*/
int v4l_vb2q_enable_media_source(struct vb2_queue *q);
+/**
+ * v4l2_pipeline_inherit_controls - Add the v4l2 controls from all
+ * subdev entities in a pipeline to
+ * the given video device.
+ * @vfd: the video device
+ * @start_entity: Starting entity
+ */
+int __v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity);
+int v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity);
/**
* v4l2_pipeline_pm_use - Update the use count of an entity
@@ -231,6 +242,20 @@ static inline int v4l_vb2q_enable_media_source(struct vb2_queue *q)
return 0;
}
+static inline int __v4l2_pipeline_inherit_controls(
+ struct video_device *vfd,
+ struct media_entity *start_entity)
+ struct video_device *vfd,
+ struct media_entity *start_entity)
{
return 0;
--
2.7.4
all subdev entities in a pipeline to a given video device.
drivers/media/v4l2-core/v4l2-mc.c | 48 +++++++++++++++++++++++++++++++++++++++
include/media/v4l2-mc.h | 25 ++++++++++++++++++++
2 files changed, 73 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 303980b..09d4d97 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -22,6 +22,7 @@
#include <linux/usb.h>
#include <media/media-device.h>
#include <media/media-entity.h>
+#include <media/v4l2-ctrls.h>
#include <media/v4l2-fh.h>
#include <media/v4l2-mc.h>
#include <media/v4l2-subdev.h>
@@ -238,6 +239,53 @@ int v4l_vb2q_enable_media_source(struct vb2_queue *q)
}
EXPORT_SYMBOL_GPL(v4l_vb2q_enable_media_source);
+int __v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ struct media_device *mdev = start_entity->graph_obj.mdev;
+ struct media_entity *entity;
+ struct media_graph graph;
+ struct v4l2_subdev *sd;
+ int ret;
+
+ ret = media_graph_walk_init(&graph, mdev);
+ if (ret)
+ return ret;
+
+ media_graph_walk_start(&graph, start_entity);
+ return ret;
+
+
+ while ((entity = media_graph_walk_next(&graph))) {
+ if (!is_media_entity_v4l2_subdev(entity))
+ continue;
+
+ sd = media_entity_to_v4l2_subdev(entity);
+
+ ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
+ sd->ctrl_handler,
+ NULL);
+ if (ret)
+ break;
+ }
+
+ media_graph_walk_cleanup(&graph);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(__v4l2_pipeline_inherit_controls);
+
+int v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ struct media_device *mdev = start_entity->graph_obj.mdev;
+ int ret;
+
+ mutex_lock(&mdev->graph_mutex);
+ ret = __v4l2_pipeline_inherit_controls(vfd, start_entity);
+ mutex_unlock(&mdev->graph_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(v4l2_pipeline_inherit_controls);
+ return ret;
+}
+
/* -----------------------------------------------------------------------------
* Pipeline power management
*
diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
index 2634d9d..9848e77 100644
--- a/include/media/v4l2-mc.h
+++ b/include/media/v4l2-mc.h
@@ -171,6 +171,17 @@ void v4l_disable_media_source(struct video_device *vdev);
*/
int v4l_vb2q_enable_media_source(struct vb2_queue *q);
+/**
+ * v4l2_pipeline_inherit_controls - Add the v4l2 controls from all
+ * subdev entities in a pipeline to
+ * the given video device.
+ * @vfd: the video device
+ * @start_entity: Starting entity
+ */
+int __v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity);
+int v4l2_pipeline_inherit_controls(struct video_device *vfd,
+ struct media_entity *start_entity);
/**
* v4l2_pipeline_pm_use - Update the use count of an entity
@@ -231,6 +242,20 @@ static inline int v4l_vb2q_enable_media_source(struct vb2_queue *q)
return 0;
}
+static inline int __v4l2_pipeline_inherit_controls(
+ struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ return 0;
+}
+
+static inline int v4l2_pipeline_inherit_controls(
+ return 0;
+}
+
+ struct video_device *vfd,
+ struct media_entity *start_entity)
+{
+ return 0;
+}
+
static inline int v4l2_pipeline_pm_use(struct media_entity *entity, int use)
+ return 0;
+}
+
{
return 0;
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:05 AM3/10/17
to
Add bindings documentation for the i.MX media driver.
Documentation/devicetree/bindings/media/imx.txt | 74 +++++++++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/imx.txt
diff --git a/Documentation/devicetree/bindings/media/imx.txt b/Documentation/devicetree/bindings/media/imx.txt
new file mode 100644
index 0000000..3059c06
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/imx.txt
@@ -0,0 +1,74 @@
+Freescale i.MX Media Video Device
+=================================
+
+Video Media Controller node
+---------------------------
+
+This is the media controller node for video capture support. It is a
+virtual device that lists the camera serial interface nodes that the
+media device will control.
+
+Required properties:
+- compatible : "fsl,imx-capture-subsystem";
+- ports : Should contain a list of phandles pointing to camera
+ sensor interface ports of IPU devices
+
+example:
+
+capture-subsystem {
+--------------
+
+This is an optional child node of the ipu_csi port nodes. If present and
+available, it enables the Frame Interval Monitor. Its properties can be
+used to modify the method in which the FIM measures frame intervals.
+Refer to Documentation/media/v4l-drivers/imx.rst for more info on the
+Frame Interval Monitor.
+
+Optional properties:
+- fsl,input-capture-channel: an input capture channel and channel flags,
+ specified as <chan flags>. The channel number
+ must be 0 or 1. The flags can be
+ IRQ_TYPE_EDGE_RISING, IRQ_TYPE_EDGE_FALLING, or
+ IRQ_TYPE_EDGE_BOTH, and specify which input
+ capture signal edge will trigger the input
+ capture event. If an input capture channel is
+ specified, the FIM will use this method to
+ measure frame intervals instead of via the EOF
+ interrupt. The input capture method is much
+ preferred over EOF as it is not subject to
+ interrupt latency errors. However it requires
+ routing the VSYNC or FIELD output signals of
+ the camera sensor to one of the i.MX input
+ capture pads (SD1_DAT0, SD1_DAT1), which also
+ gives up support for SD1.
+
+
+mipi_csi2 node
+--------------
+
+This is the device node for the MIPI CSI-2 Receiver, required for MIPI
+CSI-2 sensors.
+
+Required properties:
+- compatible : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
+- reg : physical base address and length of the register set;
+- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx
+ (the D-PHY clock), video_27m (D-PHY PLL reference
+ clock), and eim_podf;
+- clock-names : must contain "dphy", "ref", "pix";
+- port@* : five port nodes must exist, containing endpoints
+ connecting to the source and sink devices according to
+ of_graph bindings. The first port is an input port,
+ connecting with a MIPI CSI-2 source, and ports 1
+ through 4 are output ports connecting with parallel
+ bus sink endpoint nodes and correspond to the four
+ MIPI CSI-2 virtual channel outputs.
+
+Optional properties:
+- interrupts : must contain two level-triggered interrupts,
+ in order: 100 and 101;
--
2.7.4
Documentation/devicetree/bindings/media/imx.txt | 74 +++++++++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/imx.txt
diff --git a/Documentation/devicetree/bindings/media/imx.txt b/Documentation/devicetree/bindings/media/imx.txt
new file mode 100644
index 0000000..3059c06
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/imx.txt
@@ -0,0 +1,74 @@
+Freescale i.MX Media Video Device
+=================================
+
+Video Media Controller node
+---------------------------
+
+This is the media controller node for video capture support. It is a
+virtual device that lists the camera serial interface nodes that the
+media device will control.
+
+Required properties:
+- compatible : "fsl,imx-capture-subsystem";
+- ports : Should contain a list of phandles pointing to camera
+ sensor interface ports of IPU devices
+
+example:
+
+capture-subsystem {
+ compatible = "fsl,imx-capture-subsystem";
+ ports = <&ipu1_csi0>, <&ipu1_csi1>;
+};
+
+fim child node
+ ports = <&ipu1_csi0>, <&ipu1_csi1>;
+};
+
+--------------
+
+This is an optional child node of the ipu_csi port nodes. If present and
+available, it enables the Frame Interval Monitor. Its properties can be
+used to modify the method in which the FIM measures frame intervals.
+Refer to Documentation/media/v4l-drivers/imx.rst for more info on the
+Frame Interval Monitor.
+
+Optional properties:
+- fsl,input-capture-channel: an input capture channel and channel flags,
+ specified as <chan flags>. The channel number
+ must be 0 or 1. The flags can be
+ IRQ_TYPE_EDGE_RISING, IRQ_TYPE_EDGE_FALLING, or
+ IRQ_TYPE_EDGE_BOTH, and specify which input
+ capture signal edge will trigger the input
+ capture event. If an input capture channel is
+ specified, the FIM will use this method to
+ measure frame intervals instead of via the EOF
+ interrupt. The input capture method is much
+ preferred over EOF as it is not subject to
+ interrupt latency errors. However it requires
+ routing the VSYNC or FIELD output signals of
+ the camera sensor to one of the i.MX input
+ capture pads (SD1_DAT0, SD1_DAT1), which also
+ gives up support for SD1.
+
+
+mipi_csi2 node
+--------------
+
+This is the device node for the MIPI CSI-2 Receiver, required for MIPI
+CSI-2 sensors.
+
+Required properties:
+- compatible : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
+- reg : physical base address and length of the register set;
+- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx
+ (the D-PHY clock), video_27m (D-PHY PLL reference
+ clock), and eim_podf;
+- clock-names : must contain "dphy", "ref", "pix";
+- port@* : five port nodes must exist, containing endpoints
+ connecting to the source and sink devices according to
+ of_graph bindings. The first port is an input port,
+ connecting with a MIPI CSI-2 source, and ports 1
+ through 4 are output ports connecting with parallel
+ bus sink endpoint nodes and correspond to the four
+ MIPI CSI-2 virtual channel outputs.
+
+Optional properties:
+- interrupts : must contain two level-triggered interrupts,
+ in order: 100 and 101;
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:05 AM3/10/17
to
If the pads on both sides of a link specify a frame interval, then
those frame intervals should match. Create the exported function
v4l2_subdev_link_validate_frame_interval() to verify this. This
function can be called in a subdevice's media_entity_operations
or v4l2_subdev_pad_ops link_validate callbacks.
drivers/media/v4l2-core/v4l2-subdev.c | 50 +++++++++++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 10 +++++++
2 files changed, 60 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index da78497..7a0f387 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -521,6 +521,25 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
static int
+v4l2_subdev_link_validate_get_fi(struct media_pad *pad,
+
+ fi->pad = pad->index;
+ return v4l2_subdev_call(sd, video, g_frame_interval, fi);
+ }
+
+ WARN(pad->entity->function != MEDIA_ENT_F_IO_V4L,
+ "Driver bug! Wrong media entity type 0x%08x, entity %s\n",
+ pad->entity->function, pad->entity->name);
+
v4l2_subdev_link_validate_get_format(struct media_pad *pad,
struct v4l2_subdev_format *fmt)
{
@@ -540,6 +559,37 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
return -EINVAL;
}
+int v4l2_subdev_link_validate_frame_interval(struct media_link *link)
+{
+ struct v4l2_subdev_frame_interval src_fi, sink_fi;
+ unsigned long src_usec, sink_usec;
+ int rval;
+
+ rval = v4l2_subdev_link_validate_get_fi(link->source, &src_fi);
+ if (rval < 0)
+ return 0;
+
+ rval = v4l2_subdev_link_validate_get_fi(link->sink, &sink_fi);
+ if (rval < 0)
+ return 0;
+
+ if (src_fi.interval.numerator == 0 ||
+ src_fi.interval.denominator == 0 ||
+ sink_fi.interval.numerator == 0 ||
+ sink_fi.interval.denominator == 0)
+ return -EPIPE;
+
+ src_usec = DIV_ROUND_CLOSEST_ULL(
+ (u64)src_fi.interval.numerator * USEC_PER_SEC,
+ src_fi.interval.denominator);
+ sink_usec = DIV_ROUND_CLOSEST_ULL(
+ (u64)sink_fi.interval.numerator * USEC_PER_SEC,
+ sink_fi.interval.denominator);
+
+ return (src_usec != sink_usec) ? -EPIPE : 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_frame_interval);
+
int v4l2_subdev_link_validate(struct media_link *link)
{
struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 0ab1c5d..60c941d 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -929,6 +929,16 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt);
/**
+ * v4l2_subdev_link_validate_frame_interval - validates a media link
+ *
+ * @link: pointer to &struct media_link
+ *
+ * This function ensures that the frame intervals, if specified by
+ * both the source and sink subdevs of the link, are equal.
+ */
+int v4l2_subdev_link_validate_frame_interval(struct media_link *link);
+
+/**
* v4l2_subdev_link_validate - validates a media link
*
* @link: pointer to &struct media_link
--
2.7.4
those frame intervals should match. Create the exported function
v4l2_subdev_link_validate_frame_interval() to verify this. This
function can be called in a subdevice's media_entity_operations
or v4l2_subdev_pad_ops link_validate callbacks.
drivers/media/v4l2-core/v4l2-subdev.c | 50 +++++++++++++++++++++++++++++++++++
include/media/v4l2-subdev.h | 10 +++++++
2 files changed, 60 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index da78497..7a0f387 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -521,6 +521,25 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_default);
static int
+v4l2_subdev_link_validate_get_fi(struct media_pad *pad,
+ struct v4l2_subdev_frame_interval *fi)
+{
+ if (is_media_entity_v4l2_subdev(pad->entity)) {
+{
+ struct v4l2_subdev *sd =
+ media_entity_to_v4l2_subdev(pad->entity);
+
+ fi->pad = pad->index;
+ return v4l2_subdev_call(sd, video, g_frame_interval, fi);
+ }
+
+ WARN(pad->entity->function != MEDIA_ENT_F_IO_V4L,
+ "Driver bug! Wrong media entity type 0x%08x, entity %s\n",
+ pad->entity->function, pad->entity->name);
+
+ return -EINVAL;
+}
+
+static int
+}
+
v4l2_subdev_link_validate_get_format(struct media_pad *pad,
struct v4l2_subdev_format *fmt)
{
@@ -540,6 +559,37 @@ v4l2_subdev_link_validate_get_format(struct media_pad *pad,
return -EINVAL;
}
+int v4l2_subdev_link_validate_frame_interval(struct media_link *link)
+{
+ struct v4l2_subdev_frame_interval src_fi, sink_fi;
+ unsigned long src_usec, sink_usec;
+ int rval;
+
+ rval = v4l2_subdev_link_validate_get_fi(link->source, &src_fi);
+ if (rval < 0)
+ return 0;
+
+ rval = v4l2_subdev_link_validate_get_fi(link->sink, &sink_fi);
+ if (rval < 0)
+ return 0;
+
+ if (src_fi.interval.numerator == 0 ||
+ src_fi.interval.denominator == 0 ||
+ sink_fi.interval.numerator == 0 ||
+ sink_fi.interval.denominator == 0)
+ return -EPIPE;
+
+ src_usec = DIV_ROUND_CLOSEST_ULL(
+ (u64)src_fi.interval.numerator * USEC_PER_SEC,
+ src_fi.interval.denominator);
+ sink_usec = DIV_ROUND_CLOSEST_ULL(
+ (u64)sink_fi.interval.numerator * USEC_PER_SEC,
+ sink_fi.interval.denominator);
+
+ return (src_usec != sink_usec) ? -EPIPE : 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate_frame_interval);
+
int v4l2_subdev_link_validate(struct media_link *link)
{
struct v4l2_subdev *sink;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 0ab1c5d..60c941d 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -929,6 +929,16 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt);
/**
+ * v4l2_subdev_link_validate_frame_interval - validates a media link
+ *
+ * @link: pointer to &struct media_link
+ *
+ * This function ensures that the frame intervals, if specified by
+ * both the source and sink subdevs of the link, are equal.
+ */
+int v4l2_subdev_link_validate_frame_interval(struct media_link *link);
+
+/**
* v4l2_subdev_link_validate - validates a media link
*
* @link: pointer to &struct media_link
--
2.7.4
Steve Longerbeam
Mar 10, 2017, 12:10:05 AM3/10/17
to
This driver is based on ov5640_mipi.c from Freescale imx_3.10.17_1.0.0_beta
branch, modified heavily to bring forward to latest interfaces and code
cleanup.
drivers/media/i2c/Kconfig | 7 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 2239 insertions(+)
create mode 100644 drivers/media/i2c/ov5640.c
diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index cee1dae..bf67661 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -531,6 +531,13 @@ config VIDEO_OV2659
+config VIDEO_OV5640
+ tristate "OmniVision OV5640 sensor support"
+ depends on GPIOLIB && VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API
+ ---help---
+ This is a V4L2 sensor-level driver for the Omnivision
+ OV5640 camera sensor with a MIPI CSI-2 interface.
+
config VIDEO_OV7640
tristate "OmniVision OV7640 sensor support"
depends on I2C && VIDEO_V4L2
diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
index 5bc7bbe..3a9d73a 100644
--- a/drivers/media/i2c/Makefile
+++ b/drivers/media/i2c/Makefile
@@ -57,6 +57,7 @@ obj-$(CONFIG_VIDEO_VP27SMPX) += vp27smpx.o
obj-$(CONFIG_VIDEO_SONY_BTF_MPX) += sony-btf-mpx.o
obj-$(CONFIG_VIDEO_UPD64031A) += upd64031a.o
obj-$(CONFIG_VIDEO_UPD64083) += upd64083.o
+obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
obj-$(CONFIG_VIDEO_OV7670) += ov7670.o
obj-$(CONFIG_VIDEO_OV9650) += ov9650.o
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
new file mode 100644
index 0000000..2d78ec8
--- /dev/null
+++ b/drivers/media/i2c/ov5640.c
@@ -0,0 +1,2231 @@
+/*
+ * Copyright (C) 2011-2013 Freescale Semiconductor, Inc. All Rights Reserved.
+ * Copyright (C) 2014-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+#include <linux/ctype.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/i2c.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/gpio/consumer.h>
+#include <linux/regulator/consumer.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-of.h>
+#include <media/v4l2-subdev.h>
+
+/* min/typical/max system clock (xclk) frequencies */
+#define OV5640_XCLK_MIN 6000000
+#define OV5640_XCLK_MAX 24000000
+
+/*
+ * FIXME: there is no subdev API to set the MIPI CSI-2
+ * virtual channel yet, so this is hardcoded for now.
+ */
+#define OV5640_MIPI_VC 1
+
+#define OV5640_DEFAULT_SLAVE_ID 0x3c
+
+#define OV5640_REG_CHIP_ID 0x300a
+#define OV5640_REG_PAD_OUTPUT00 0x3019
+#define OV5640_REG_SC_PLL_CTRL0 0x3034
+#define OV5640_REG_SC_PLL_CTRL1 0x3035
+#define OV5640_REG_SC_PLL_CTRL2 0x3036
+#define OV5640_REG_SC_PLL_CTRL3 0x3037
+#define OV5640_REG_SLAVE_ID 0x3100
+#define OV5640_REG_SYS_ROOT_DIVIDER 0x3108
+#define OV5640_REG_AWB_R_GAIN 0x3400
+#define OV5640_REG_AWB_G_GAIN 0x3402
+#define OV5640_REG_AWB_B_GAIN 0x3404
+#define OV5640_REG_AWB_MANUAL_CTRL 0x3406
+#define OV5640_REG_AEC_PK_EXPOSURE_HI 0x3500
+#define OV5640_REG_AEC_PK_EXPOSURE_MED 0x3501
+#define OV5640_REG_AEC_PK_EXPOSURE_LO 0x3502
+#define OV5640_REG_AEC_PK_MANUAL 0x3503
+#define OV5640_REG_AEC_PK_REAL_GAIN 0x350a
+#define OV5640_REG_AEC_PK_VTS 0x350c
+#define OV5640_REG_TIMING_HTS 0x380c
+#define OV5640_REG_TIMING_VTS 0x380e
+#define OV5640_REG_TIMING_TC_REG21 0x3821
+#define OV5640_REG_AEC_CTRL00 0x3a00
+#define OV5640_REG_AEC_B50_STEP 0x3a08
+#define OV5640_REG_AEC_B60_STEP 0x3a0a
+#define OV5640_REG_AEC_CTRL0D 0x3a0d
+#define OV5640_REG_AEC_CTRL0E 0x3a0e
+#define OV5640_REG_AEC_CTRL0F 0x3a0f
+#define OV5640_REG_AEC_CTRL10 0x3a10
+#define OV5640_REG_AEC_CTRL11 0x3a11
+#define OV5640_REG_AEC_CTRL1B 0x3a1b
+#define OV5640_REG_AEC_CTRL1E 0x3a1e
+#define OV5640_REG_AEC_CTRL1F 0x3a1f
+#define OV5640_REG_HZ5060_CTRL00 0x3c00
+#define OV5640_REG_HZ5060_CTRL01 0x3c01
+#define OV5640_REG_SIGMADELTA_CTRL0C 0x3c0c
+#define OV5640_REG_FRAME_CTRL01 0x4202
+#define OV5640_REG_MIPI_CTRL00 0x4800
+#define OV5640_REG_DEBUG_MODE 0x4814
+#define OV5640_REG_PRE_ISP_TEST_SET1 0x503d
+#define OV5640_REG_SDE_CTRL0 0x5580
+#define OV5640_REG_SDE_CTRL1 0x5581
+#define OV5640_REG_SDE_CTRL3 0x5583
+#define OV5640_REG_SDE_CTRL4 0x5584
+#define OV5640_REG_SDE_CTRL5 0x5585
+#define OV5640_REG_AVG_READOUT 0x56a1
+
+enum ov5640_mode_id {
+ OV5640_MODE_QCIF_176_144 = 0,
+ OV5640_MODE_QVGA_320_240,
+ OV5640_MODE_VGA_640_480,
+ OV5640_MODE_NTSC_720_480,
+ OV5640_MODE_PAL_720_576,
+ OV5640_MODE_XGA_1024_768,
+ OV5640_MODE_720P_1280_720,
+ OV5640_MODE_1080P_1920_1080,
+ OV5640_MODE_QSXGA_2592_1944,
+ OV5640_NUM_MODES,
+};
+
+enum ov5640_frame_rate {
+ OV5640_15_FPS = 0,
+ OV5640_30_FPS,
+ OV5640_NUM_FRAMERATES,
+};
+
+static const int ov5640_framerates[] = {
+ [OV5640_15_FPS] = 15,
+ [OV5640_30_FPS] = 30,
+};
+
+/* regulator supplies */
+static const char * const ov5640_supply_name[] = {
+ "DOVDD", /* Digital I/O (1.8V) suppply */
+ "DVDD", /* Digital Core (1.5V) supply */
+ "AVDD", /* Analog (2.8V) supply */
+};
+
+#define OV5640_NUM_SUPPLIES ARRAY_SIZE(ov5640_supply_name)
+
+/*
+ * image size under 1280 * 960 are SUBSAMPLING
+ * image size upper 1280 * 960 are SCALING
+ */
+enum ov5640_downsize_mode {
+ SUBSAMPLING,
+ SCALING,
+};
+
+struct reg_value {
+ u16 reg_addr;
+ u8 val;
+ u8 mask;
+ u32 delay_ms;
+};
+
+struct ov5640_mode_info {
+ enum ov5640_mode_id id;
+ enum ov5640_downsize_mode dn_mode;
+ u32 width;
+ u32 height;
+ const struct reg_value *reg_data;
+ u32 reg_data_size;
+};
+
+struct ov5640_ctrls {
+ struct v4l2_ctrl_handler handler;
+ struct {
+ struct v4l2_ctrl *auto_exp;
+ struct v4l2_ctrl *exposure;
+ };
+ struct {
+ struct v4l2_ctrl *auto_wb;
+ struct v4l2_ctrl *blue_balance;
+ struct v4l2_ctrl *red_balance;
+ };
+ struct {
+ struct v4l2_ctrl *auto_gain;
+ struct v4l2_ctrl *gain;
+ };
+ struct v4l2_ctrl *brightness;
+ struct v4l2_ctrl *saturation;
+ struct v4l2_ctrl *contrast;
+ struct v4l2_ctrl *hue;
+ struct v4l2_ctrl *test_pattern;
+};
+
+struct ov5640_dev {
+ struct i2c_client *i2c_client;
+ struct v4l2_subdev sd;
+ struct media_pad pad;
+ struct v4l2_of_endpoint ep; /* the parsed DT endpoint info */
+ struct clk *xclk; /* system clock to OV5640 */
+ u32 xclk_freq;
+
+ struct regulator_bulk_data supplies[OV5640_NUM_SUPPLIES];
+ struct gpio_desc *reset_gpio;
+ struct gpio_desc *pwdn_gpio;
+
+ struct v4l2_mbus_framefmt fmt;
+ struct v4l2_captureparm streamcap;
+
+ const struct ov5640_mode_info *current_mode;
+ enum ov5640_frame_rate current_fr;
+
+ struct ov5640_ctrls ctrls;
+
+ u32 prev_sysclk, prev_hts;
+ u32 ae_low, ae_high, ae_target;
+
+ bool pending_mode_change;
+ bool streaming;
+};
+
+static inline struct ov5640_dev *to_ov5640_dev(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct ov5640_dev, sd);
+}
+
+static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
+{
+ return &container_of(ctrl->handler, struct ov5640_dev,
+ ctrls.handler)->sd;
+}
+
+/*
+ * FIXME: all of these register tables are likely filled with
+ * entries that set the register to their power-on default values,
+ * and which are otherwise not touched by this driver. Those entries
+ * should be identified and removed to speed register load time
+ * over i2c.
+ */
+
+static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
+
+ {0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
+ {0x3103, 0x03, 0, 0}, {0x3017, 0x00, 0, 0}, {0x3018, 0x00, 0, 0},
+ {0x3034, 0x18, 0, 0}, {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0},
+ {0x3037, 0x13, 0, 0}, {0x3108, 0x01, 0, 0}, {0x3630, 0x36, 0, 0},
+ {0x3631, 0x0e, 0, 0}, {0x3632, 0xe2, 0, 0}, {0x3633, 0x12, 0, 0},
+ {0x3621, 0xe0, 0, 0}, {0x3704, 0xa0, 0, 0}, {0x3703, 0x5a, 0, 0},
+ {0x3715, 0x78, 0, 0}, {0x3717, 0x01, 0, 0}, {0x370b, 0x60, 0, 0},
+ {0x3705, 0x1a, 0, 0}, {0x3905, 0x02, 0, 0}, {0x3906, 0x10, 0, 0},
+ {0x3901, 0x0a, 0, 0}, {0x3731, 0x12, 0, 0}, {0x3600, 0x08, 0, 0},
+ {0x3601, 0x33, 0, 0}, {0x302d, 0x60, 0, 0}, {0x3620, 0x52, 0, 0},
+ {0x371b, 0x20, 0, 0}, {0x471c, 0x50, 0, 0}, {0x3a13, 0x43, 0, 0},
+ {0x3a18, 0x00, 0, 0}, {0x3a19, 0xf8, 0, 0}, {0x3635, 0x13, 0, 0},
+ {0x3636, 0x03, 0, 0}, {0x3634, 0x40, 0, 0}, {0x3622, 0x01, 0, 0},
+ {0x3c01, 0xa4, 0, 0}, {0x3c04, 0x28, 0, 0}, {0x3c05, 0x98, 0, 0},
+ {0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x3000, 0x00, 0, 0},
+ {0x3002, 0x1c, 0, 0}, {0x3004, 0xff, 0, 0}, {0x3006, 0xc3, 0, 0},
+ {0x300e, 0x45, 0, 0}, {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
+ {0x501f, 0x00, 0, 0}, {0x4713, 0x03, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x440e, 0x00, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x4837, 0x0a, 0, 0}, {0x4800, 0x04, 0, 0}, {0x3824, 0x02, 0, 0},
+ {0x5000, 0xa7, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x5180, 0xff, 0, 0},
+ {0x5181, 0xf2, 0, 0}, {0x5182, 0x00, 0, 0}, {0x5183, 0x14, 0, 0},
+ {0x5184, 0x25, 0, 0}, {0x5185, 0x24, 0, 0}, {0x5186, 0x09, 0, 0},
+ {0x5187, 0x09, 0, 0}, {0x5188, 0x09, 0, 0}, {0x5189, 0x88, 0, 0},
+ {0x518a, 0x54, 0, 0}, {0x518b, 0xee, 0, 0}, {0x518c, 0xb2, 0, 0},
+ {0x518d, 0x50, 0, 0}, {0x518e, 0x34, 0, 0}, {0x518f, 0x6b, 0, 0},
+ {0x5190, 0x46, 0, 0}, {0x5191, 0xf8, 0, 0}, {0x5192, 0x04, 0, 0},
+ {0x5193, 0x70, 0, 0}, {0x5194, 0xf0, 0, 0}, {0x5195, 0xf0, 0, 0},
+ {0x5196, 0x03, 0, 0}, {0x5197, 0x01, 0, 0}, {0x5198, 0x04, 0, 0},
+ {0x5199, 0x6c, 0, 0}, {0x519a, 0x04, 0, 0}, {0x519b, 0x00, 0, 0},
+ {0x519c, 0x09, 0, 0}, {0x519d, 0x2b, 0, 0}, {0x519e, 0x38, 0, 0},
+ {0x5381, 0x1e, 0, 0}, {0x5382, 0x5b, 0, 0}, {0x5383, 0x08, 0, 0},
+ {0x5384, 0x0a, 0, 0}, {0x5385, 0x7e, 0, 0}, {0x5386, 0x88, 0, 0},
+ {0x5387, 0x7c, 0, 0}, {0x5388, 0x6c, 0, 0}, {0x5389, 0x10, 0, 0},
+ {0x538a, 0x01, 0, 0}, {0x538b, 0x98, 0, 0}, {0x5300, 0x08, 0, 0},
+ {0x5301, 0x30, 0, 0}, {0x5302, 0x10, 0, 0}, {0x5303, 0x00, 0, 0},
+ {0x5304, 0x08, 0, 0}, {0x5305, 0x30, 0, 0}, {0x5306, 0x08, 0, 0},
+ {0x5307, 0x16, 0, 0}, {0x5309, 0x08, 0, 0}, {0x530a, 0x30, 0, 0},
+ {0x530b, 0x04, 0, 0}, {0x530c, 0x06, 0, 0}, {0x5480, 0x01, 0, 0},
+ {0x5481, 0x08, 0, 0}, {0x5482, 0x14, 0, 0}, {0x5483, 0x28, 0, 0},
+ {0x5484, 0x51, 0, 0}, {0x5485, 0x65, 0, 0}, {0x5486, 0x71, 0, 0},
+ {0x5487, 0x7d, 0, 0}, {0x5488, 0x87, 0, 0}, {0x5489, 0x91, 0, 0},
+ {0x548a, 0x9a, 0, 0}, {0x548b, 0xaa, 0, 0}, {0x548c, 0xb8, 0, 0},
+ {0x548d, 0xcd, 0, 0}, {0x548e, 0xdd, 0, 0}, {0x548f, 0xea, 0, 0},
+ {0x5490, 0x1d, 0, 0}, {0x5580, 0x02, 0, 0}, {0x5583, 0x40, 0, 0},
+ {0x5584, 0x10, 0, 0}, {0x5589, 0x10, 0, 0}, {0x558a, 0x00, 0, 0},
+ {0x558b, 0xf8, 0, 0}, {0x5800, 0x23, 0, 0}, {0x5801, 0x14, 0, 0},
+ {0x5802, 0x0f, 0, 0}, {0x5803, 0x0f, 0, 0}, {0x5804, 0x12, 0, 0},
+ {0x5805, 0x26, 0, 0}, {0x5806, 0x0c, 0, 0}, {0x5807, 0x08, 0, 0},
+ {0x5808, 0x05, 0, 0}, {0x5809, 0x05, 0, 0}, {0x580a, 0x08, 0, 0},
+ {0x580b, 0x0d, 0, 0}, {0x580c, 0x08, 0, 0}, {0x580d, 0x03, 0, 0},
+ {0x580e, 0x00, 0, 0}, {0x580f, 0x00, 0, 0}, {0x5810, 0x03, 0, 0},
+ {0x5811, 0x09, 0, 0}, {0x5812, 0x07, 0, 0}, {0x5813, 0x03, 0, 0},
+ {0x5814, 0x00, 0, 0}, {0x5815, 0x01, 0, 0}, {0x5816, 0x03, 0, 0},
+ {0x5817, 0x08, 0, 0}, {0x5818, 0x0d, 0, 0}, {0x5819, 0x08, 0, 0},
+ {0x581a, 0x05, 0, 0}, {0x581b, 0x06, 0, 0}, {0x581c, 0x08, 0, 0},
+ {0x581d, 0x0e, 0, 0}, {0x581e, 0x29, 0, 0}, {0x581f, 0x17, 0, 0},
+ {0x5820, 0x11, 0, 0}, {0x5821, 0x11, 0, 0}, {0x5822, 0x15, 0, 0},
+ {0x5823, 0x28, 0, 0}, {0x5824, 0x46, 0, 0}, {0x5825, 0x26, 0, 0},
+ {0x5826, 0x08, 0, 0}, {0x5827, 0x26, 0, 0}, {0x5828, 0x64, 0, 0},
+ {0x5829, 0x26, 0, 0}, {0x582a, 0x24, 0, 0}, {0x582b, 0x22, 0, 0},
+ {0x582c, 0x24, 0, 0}, {0x582d, 0x24, 0, 0}, {0x582e, 0x06, 0, 0},
+ {0x582f, 0x22, 0, 0}, {0x5830, 0x40, 0, 0}, {0x5831, 0x42, 0, 0},
+ {0x5832, 0x24, 0, 0}, {0x5833, 0x26, 0, 0}, {0x5834, 0x24, 0, 0},
+ {0x5835, 0x22, 0, 0}, {0x5836, 0x22, 0, 0}, {0x5837, 0x26, 0, 0},
+ {0x5838, 0x44, 0, 0}, {0x5839, 0x24, 0, 0}, {0x583a, 0x26, 0, 0},
+ {0x583b, 0x28, 0, 0}, {0x583c, 0x42, 0, 0}, {0x583d, 0xce, 0, 0},
+ {0x5025, 0x00, 0, 0}, {0x3a0f, 0x30, 0, 0}, {0x3a10, 0x28, 0, 0},
+ {0x3a1b, 0x30, 0, 0}, {0x3a1e, 0x26, 0, 0}, {0x3a11, 0x60, 0, 0},
+ {0x3a1f, 0x14, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3c00, 0x04, 0, 300},
+};
+
+static const struct reg_value ov5640_setting_30fps_VGA_640_480[] = {
+
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x04, 0, 0}, {0x380f, 0x38, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x0e, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3503, 0x00, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_VGA_640_480[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_XGA_1024_768[] = {
+
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x04, 0, 0}, {0x380f, 0x38, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x0e, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3503, 0x00, 0, 0},
+ {0x3808, 0x04, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x03, 0, 0},
+ {0x380b, 0x00, 0, 0}, {0x3035, 0x12, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_XGA_1024_768[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3808, 0x04, 0, 0},
+ {0x3809, 0x00, 0, 0}, {0x380a, 0x03, 0, 0}, {0x380b, 0x00, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_QVGA_320_240[] = {
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x01, 0, 0}, {0x3809, 0x40, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0xf0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_QVGA_320_240[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x01, 0, 0}, {0x3809, 0x40, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0xf0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_QCIF_176_144[] = {
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x00, 0, 0}, {0x3809, 0xb0, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0x90, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+static const struct reg_value ov5640_setting_15fps_QCIF_176_144[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x00, 0, 0}, {0x3809, 0xb0, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0x90, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_NTSC_720_480[] = {
+ {0x3035, 0x12, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_NTSC_720_480[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_PAL_720_576[] = {
+ {0x3035, 0x12, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0x40, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_PAL_720_576[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0x40, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_720P_1280_720[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
+ {0x3808, 0x05, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0xd0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x64, 0, 0},
+ {0x380e, 0x02, 0, 0}, {0x380f, 0xe4, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
+ {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
+ {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0},
+ {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x02, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0},
+ {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0}, {0x4005, 0x1a, 0, 0},
+ {0x3008, 0x02, 0, 0}, {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_720P_1280_720[] = {
+ {0x3035, 0x41, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
+ {0x3808, 0x05, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0xd0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x64, 0, 0},
+ {0x380e, 0x02, 0, 0}, {0x380f, 0xe4, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
+ {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
+ {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0},
+ {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x02, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0},
+ {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_1080P_1920_1080[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, {0x3035, 0x11, 0, 0},
+ {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
+ {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
+ {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, {0x3808, 0x07, 0, 0},
+ {0x3809, 0x80, 0, 0}, {0x380a, 0x04, 0, 0}, {0x380b, 0x38, 0, 0},
+ {0x380c, 0x09, 0, 0}, {0x380d, 0xc4, 0, 0}, {0x380e, 0x04, 0, 0},
+ {0x380f, 0x60, 0, 0}, {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
+ {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
+ {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0},
+ {0x3a15, 0x60, 0, 0}, {0x4713, 0x02, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0},
+ {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0},
+ {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_1080P_1920_1080[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, {0x3035, 0x21, 0, 0},
+ {0x3036, 0x54, 0, 1}, {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
+ {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
+ {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, {0x3808, 0x07, 0, 0},
+ {0x3809, 0x80, 0, 0}, {0x380a, 0x04, 0, 0}, {0x380b, 0x38, 0, 0},
+ {0x380c, 0x09, 0, 0}, {0x380d, 0xc4, 0, 0}, {0x380e, 0x04, 0, 0},
+ {0x380f, 0x60, 0, 0}, {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
+ {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
+ {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0},
+ {0x3a15, 0x60, 0, 0}, {0x4713, 0x02, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0},
+ {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_QSXGA_2592_1944[] = {
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
+};
+
+/* power-on sensor init reg table */
+static const struct ov5640_mode_info ov5640_mode_init_data = {
+ 0, SUBSAMPLING, 640, 480, ov5640_init_setting_30fps_VGA,
+ ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
+};
+
+static const struct ov5640_mode_info
+ov5640_mode_data[OV5640_NUM_FRAMERATES][OV5640_NUM_MODES] = {
+ {
+ {OV5640_MODE_QCIF_176_144, SUBSAMPLING, 176, 144,
+ ov5640_setting_15fps_QCIF_176_144,
+ ARRAY_SIZE(ov5640_setting_15fps_QCIF_176_144)},
+ {OV5640_MODE_QVGA_320_240, SUBSAMPLING, 320, 240,
+ ov5640_setting_15fps_QVGA_320_240,
+ ARRAY_SIZE(ov5640_setting_15fps_QVGA_320_240)},
+ {OV5640_MODE_VGA_640_480, SUBSAMPLING, 640, 480,
+ ov5640_setting_15fps_VGA_640_480,
+ ARRAY_SIZE(ov5640_setting_15fps_VGA_640_480)},
+ {OV5640_MODE_NTSC_720_480, SUBSAMPLING, 720, 480,
+ ov5640_setting_15fps_NTSC_720_480,
+ ARRAY_SIZE(ov5640_setting_15fps_NTSC_720_480)},
+ {OV5640_MODE_PAL_720_576, SUBSAMPLING, 720, 576,
+ ov5640_setting_15fps_PAL_720_576,
+ ARRAY_SIZE(ov5640_setting_15fps_PAL_720_576)},
+ {OV5640_MODE_XGA_1024_768, SUBSAMPLING, 1024, 768,
+ ov5640_setting_15fps_XGA_1024_768,
+ ARRAY_SIZE(ov5640_setting_15fps_XGA_1024_768)},
+ {OV5640_MODE_720P_1280_720, SUBSAMPLING, 1280, 720,
+ ov5640_setting_15fps_720P_1280_720,
+ ARRAY_SIZE(ov5640_setting_15fps_720P_1280_720)},
+ {OV5640_MODE_1080P_1920_1080, SCALING, 1920, 1080,
+ ov5640_setting_15fps_1080P_1920_1080,
+ ARRAY_SIZE(ov5640_setting_15fps_1080P_1920_1080)},
+ {OV5640_MODE_QSXGA_2592_1944, SCALING, 2592, 1944,
+ ov5640_setting_15fps_QSXGA_2592_1944,
+ ARRAY_SIZE(ov5640_setting_15fps_QSXGA_2592_1944)},
+ }, {
+ {OV5640_MODE_QCIF_176_144, SUBSAMPLING, 176, 144,
+ ov5640_setting_30fps_QCIF_176_144,
+ ARRAY_SIZE(ov5640_setting_30fps_QCIF_176_144)},
+ {OV5640_MODE_QVGA_320_240, SUBSAMPLING, 320, 240,
+ ov5640_setting_30fps_QVGA_320_240,
+ ARRAY_SIZE(ov5640_setting_30fps_QVGA_320_240)},
+ {OV5640_MODE_VGA_640_480, SUBSAMPLING, 640, 480,
+ ov5640_setting_30fps_VGA_640_480,
+ ARRAY_SIZE(ov5640_setting_30fps_VGA_640_480)},
+ {OV5640_MODE_NTSC_720_480, SUBSAMPLING, 720, 480,
+ ov5640_setting_30fps_NTSC_720_480,
+ ARRAY_SIZE(ov5640_setting_30fps_NTSC_720_480)},
+ {OV5640_MODE_PAL_720_576, SUBSAMPLING, 720, 576,
+ ov5640_setting_30fps_PAL_720_576,
+ ARRAY_SIZE(ov5640_setting_30fps_PAL_720_576)},
+ {OV5640_MODE_XGA_1024_768, SUBSAMPLING, 1024, 768,
+ ov5640_setting_30fps_XGA_1024_768,
+ ARRAY_SIZE(ov5640_setting_30fps_XGA_1024_768)},
+ {OV5640_MODE_720P_1280_720, SUBSAMPLING, 1280, 720,
+ ov5640_setting_30fps_720P_1280_720,
+ ARRAY_SIZE(ov5640_setting_30fps_720P_1280_720)},
+ {OV5640_MODE_1080P_1920_1080, SCALING, 1920, 1080,
+ ov5640_setting_30fps_1080P_1920_1080,
+ ARRAY_SIZE(ov5640_setting_30fps_1080P_1920_1080)},
+ {OV5640_MODE_QSXGA_2592_1944, -1, 0, 0, NULL, 0},
+ },
+};
+
+static int ov5640_init_slave_id(struct ov5640_dev *sensor)
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg;
+ u8 buf[3];
+ int ret;
+
+ if (client->addr == OV5640_DEFAULT_SLAVE_ID)
+ return 0;
+
+ buf[0] = OV5640_REG_SLAVE_ID >> 8;
+ buf[1] = OV5640_REG_SLAVE_ID & 0xff;
+ buf[2] = client->addr << 1;
+
+ msg.addr = OV5640_DEFAULT_SLAVE_ID;
+ msg.flags = 0;
+ msg.buf = buf;
+ msg.len = sizeof(buf);
+
+ ret = i2c_transfer(client->adapter, &msg, 1);
+ if (ret < 0) {
+ dev_err(&client->dev, "%s: failed with %d\n", __func__, ret);
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg;
+ u8 buf[3];
+ int ret;
+
+ buf[0] = reg >> 8;
+ buf[1] = reg & 0xff;
+ buf[2] = val;
+
+ msg.addr = client->addr;
+ msg.flags = client->flags;
+ msg.buf = buf;
+ msg.len = sizeof(buf);
+
+ ret = i2c_transfer(client->adapter, &msg, 1);
+ if (ret < 0) {
+ v4l2_err(&sensor->sd, "%s: error: reg=%x, val=%x\n",
+ __func__, reg, val);
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg[2];
+ u8 buf[2];
+ int ret;
+
+ buf[0] = reg >> 8;
+ buf[1] = reg & 0xff;
+
+ msg[0].addr = client->addr;
+ msg[0].flags = client->flags;
+ msg[0].buf = buf;
+ msg[0].len = sizeof(buf);
+
+ msg[1].addr = client->addr;
+ msg[1].flags = client->flags | I2C_M_RD;
+ msg[1].buf = buf;
+ msg[1].len = 1;
+
+ ret = i2c_transfer(client->adapter, msg, 2);
+ if (ret < 0)
+ return ret;
+
+ *val = buf[0];
+{
+ u8 hi, lo;
+
+ ov5640_read_reg(sensor, reg, &hi);
+ ov5640_read_reg(sensor, reg+1, &lo);
+
+ *val = ((u16)hi << 8) | (u16)lo;
+{
+ ov5640_write_reg(sensor, reg, val >> 8);
+ ov5640_write_reg(sensor, reg + 1, val & 0xff);
+ u8 mask, u8 val)
+{
+ u8 readval;
+
+ ov5640_read_reg(sensor, reg, &readval);
+
+ readval &= ~mask;
+ val &= mask;
+ val |= readval;
+
+ ov5640_write_reg(sensor, reg, val);
+static int ov5640_load_regs(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *mode)
+{
+ const struct reg_value *regs = mode->reg_data;
+ unsigned int i;
+ u32 delay_ms;
+ u16 reg_addr;
+ u8 mask, val;
+
+ for (i = 0; i < mode->reg_data_size; ++i, ++regs) {
+ delay_ms = regs->delay_ms;
+ reg_addr = regs->reg_addr;
+ val = regs->val;
+ mask = regs->mask;
+
+ if (mask)
+ ov5640_mod_reg(sensor, reg_addr, mask, val);
+ else
+ ov5640_write_reg(sensor, reg_addr, val);
+ if (delay_ms)
+ usleep_range(1000*delay_ms, 1000*delay_ms+100);
+ }
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(0), BIT(0));
+ ov5640_write_reg16(sensor, OV5640_REG_SDE_CTRL1, value);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(0), 0);
+ }
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(2), BIT(2));
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL5, value & 0xff);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(2), 0);
+ }
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(1), BIT(1));
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL3, value & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL4, value & 0xff);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(1), 0);
+ }
+
+{
+ ov5640_mod_reg(sensor, OV5640_REG_AWB_MANUAL_CTRL,
+ BIT(0), awb ? 0 : 1);
+
+ if (!awb) {
+ u16 red = (u16)sensor->ctrls.red_balance->val;
+ u16 blue = (u16)sensor->ctrls.blue_balance->val;
+
+ ov5640_write_reg16(sensor, OV5640_REG_AWB_R_GAIN, red);
+ ov5640_write_reg16(sensor, OV5640_REG_AWB_B_GAIN, blue);
+ }
+
+{
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+ bool auto_exposure = (exp == V4L2_EXPOSURE_AUTO);
+
+ if (ctrls->auto_exp->is_new) {
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ BIT(0), auto_exposure ? 0 : BIT(0));
+ }
+
+ if (!auto_exposure && ctrls->exposure->is_new) {
+ u16 max_exp;
+
+ ov5640_read_reg16(sensor, OV5640_REG_AEC_PK_VTS, &max_exp);
+ if (ctrls->exposure->val < max_exp) {
+ u32 exposure = ctrls->exposure->val << 4;
+
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_LO,
+ exposure & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_MED,
+ (exposure >> 8) & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_HI,
+ (exposure >> 16) & 0x0f);
+ }
+ }
+
+static int ov5640_get_exposure(struct ov5640_dev *sensor)
+{
+ u8 temp;
+ int exp;
+
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_HI, &temp);
+ exp = ((int)temp & 0x0f) << 16;
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_MED, &temp);
+ exp |= ((int)temp << 8);
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_LO, &temp);
+ exp |= (int)temp;
+
+ return exp >> 4;
+}
+
+static int ov5640_set_gain(struct ov5640_dev *sensor, int auto_gain)
+{
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+
+ if (ctrls->auto_gain->is_new) {
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ BIT(1), ctrls->auto_gain->val ? 0 : BIT(1));
+ }
+
+ if (!auto_gain && ctrls->gain->is_new) {
+ u16 gain = (u16)ctrls->gain->val;
+
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_PK_REAL_GAIN,
+ gain & 0x3ff);
+ }
+
+{
+ u16 gain;
+
+ ov5640_read_reg16(sensor, OV5640_REG_AEC_PK_REAL_GAIN, &gain);
+
+ return gain & 0x3ff;
+}
+
+static int ov5640_set_agc_aec(struct ov5640_dev *sensor, bool on)
+{
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ 0x3, on ? 0 : 0x3);
+{
+ ov5640_mod_reg(sensor, OV5640_REG_PRE_ISP_TEST_SET1,
+ 0xa4, value ? 0xa4 : 0);
+{
+ ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
+ on ? 0 : BIT(5));
+ ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
+ on ? 0x00 : 0x70);
+ ov5640_write_reg(sensor, OV5640_REG_FRAME_CTRL01,
+ on ? 0x00 : 0x0f);
+{
+ /* calculate sysclk */
+ u32 xvclk = sensor->xclk_freq / 10000;
+ u32 multiplier, prediv, VCO, sysdiv, pll_rdiv;
+ u32 sclk_rdiv_map[] = {1, 2, 4, 8};
+ u32 bit_div2x = 1, sclk_rdiv, sysclk;
+ u8 temp1, temp2;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL0, &temp1);
+ temp2 = temp1 & 0x0f;
+ if (temp2 == 8 || temp2 == 10)
+ bit_div2x = temp2 / 2;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL1, &temp1);
+ sysdiv = temp1 >> 4;
+ if (sysdiv == 0)
+ sysdiv = 16;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL2, &temp1);
+ multiplier = temp1;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL3, &temp1);
+ prediv = temp1 & 0x0f;
+ pll_rdiv = ((temp1 >> 4) & 0x01) + 1;
+
+ ov5640_read_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, &temp1);
+ temp2 = temp1 & 0x03;
+ sclk_rdiv = sclk_rdiv_map[temp2];
+
+ VCO = xvclk * multiplier / prediv;
+
+ sysclk = VCO / sysdiv / pll_rdiv * 2 / bit_div2x / sclk_rdiv;
+
+ return sysclk;
+}
+
+static int ov5640_set_night_mode(struct ov5640_dev *sensor)
+{
+ /* read HTS from register settings */
+ u8 mode;
+
+ ov5640_read_reg(sensor, OV5640_REG_AEC_CTRL00, &mode);
+ mode &= 0xfb;
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL00, mode);
+{
+ /* read HTS from register settings */
+ u16 hts;
+
+ ov5640_read_reg16(sensor, OV5640_REG_TIMING_HTS, &hts);
+ return hts;
+}
+
+static int ov5640_get_vts(struct ov5640_dev *sensor)
+{
+ u16 vts;
+
+ ov5640_read_reg16(sensor, OV5640_REG_TIMING_VTS, &vts);
+ return vts;
+}
+
+static int ov5640_set_vts(struct ov5640_dev *sensor, int vts)
+{
+ ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, vts);
+{
+ /* get banding filter value */
+ u8 temp, temp1;
+ int light_freq = 0;
+
+ ov5640_read_reg(sensor, OV5640_REG_HZ5060_CTRL01, &temp);
+
+ if (temp & 0x80) {
+ /* manual */
+ ov5640_read_reg(sensor, OV5640_REG_HZ5060_CTRL00, &temp1);
+ if (temp1 & 0x04) {
+ /* 50Hz */
+ light_freq = 50;
+ } else {
+ /* 60Hz */
+ light_freq = 60;
+ }
+ } else {
+ /* auto */
+ ov5640_read_reg(sensor, OV5640_REG_SIGMADELTA_CTRL0C,
+ &temp1);
+ if (temp1 & 0x01) {
+ /* 50Hz */
+ light_freq = 50;
+ } else {
+ /* 60Hz */
+ }
+ }
+
+ return light_freq;
+}
+
+static int ov5640_set_bandingfilter(struct ov5640_dev *sensor)
+{
+ u32 band_step60, max_band60, band_step50, max_band50, prev_vts;
+ int ret;
+
+ /* read preview PCLK */
+ ret = ov5640_get_sysclk(sensor);
+ /* read preview HTS */
+ ret = ov5640_get_hts(sensor);
+
+ /* read preview VTS */
+ ret = ov5640_get_vts(sensor);
+
+ /* calculate banding filter */
+ /* 60Hz */
+ band_step60 = sensor->prev_sysclk * 100 / sensor->prev_hts * 100/120;
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_B60_STEP, band_step60);
+
+ max_band60 = (int)((prev_vts-4)/band_step60);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0D, max_band60);
+
+ /* 50Hz */
+ band_step50 = sensor->prev_sysclk * 100 / sensor->prev_hts;
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_B50_STEP, band_step50);
+
+ max_band50 = (int)((prev_vts-4)/band_step50);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0E, max_band50);
+
+{
+ /* stable in high */
+ u32 fast_high, fast_low;
+
+ sensor->ae_low = target * 23 / 25; /* 0.92 */
+ sensor->ae_high = target * 27 / 25; /* 1.08 */
+
+ fast_high = sensor->ae_high << 1;
+ if (fast_high > 255)
+ fast_high = 255;
+
+ fast_low = sensor->ae_low >> 1;
+
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0F, sensor->ae_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL10, sensor->ae_low);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1B, sensor->ae_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1E, sensor->ae_low);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL11, fast_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1F, fast_low);
+
+{
+ u8 temp;
+
+ ov5640_read_reg(sensor, OV5640_REG_TIMING_TC_REG21, &temp);
+ temp &= 0xfe;
+
+ return temp ? 1 : 0;
+}
+
+static int ov5640_set_virtual_channel(struct ov5640_dev *sensor)
+{
+ u8 temp, channel = OV5640_MIPI_VC;
+
+ ov5640_read_reg(sensor, OV5640_REG_DEBUG_MODE, &temp);
+ temp &= ~(3 << 6);
+ temp |= (channel << 6);
+ ov5640_write_reg(sensor, OV5640_REG_DEBUG_MODE, temp);
+
+ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
+ int width, int height, bool nearest)
+{
+ const struct ov5640_mode_info *mode = NULL;
+ int i;
+
+ for (i = OV5640_NUM_MODES - 1; i >= 0; i--) {
+ mode = &ov5640_mode_data[fr][i];
+
+ if (!mode->reg_data)
+ continue;
+
+ if ((nearest && mode->width <= width &&
+ mode->height <= height) ||
+ (!nearest && mode->width == width &&
+ mode->height == height))
+ break;
+ }
+
+ if (nearest && i < 0)
+ mode = &ov5640_mode_data[fr][0];
+
+ return mode;
+}
+
+/*
+ * sensor changes between scaling and subsampling, go through
+ * exposure calculation
+ */
+static int ov5640_set_mode_exposure_calc(
+ struct ov5640_dev *sensor, const struct ov5640_mode_info *mode)
+{
+ u32 prev_shutter, prev_gain16;
+ u32 cap_shutter, cap_gain16;
+ u32 cap_sysclk, cap_hts, cap_vts;
+ u32 light_freq, cap_bandfilt, cap_maxband;
+ u32 cap_gain16_shutter;
+ u8 average;
+ int ret;
+
+ if (mode->reg_data == NULL)
+ return -EINVAL;
+
+ /* turn off AE/AG */
+ ret = ov5640_set_agc_aec(sensor, false);
+ if (ret < 0)
+ return ret;
+
+ /* read preview shutter */
+ ret = ov5640_get_exposure(sensor);
+ ret = ov5640_binning_on(sensor);
+ mode->id != OV5640_MODE_1080P_1920_1080)
+ prev_shutter *= 2;
+
+ /* read preview gain */
+ ret = ov5640_get_gain(sensor);
+
+ /* get average */
+ ov5640_read_reg(sensor, OV5640_REG_AVG_READOUT, &average);
+
+ /* turn off night mode for capture */
+ ret = ov5640_set_night_mode(sensor);
+ if (ret < 0)
+ return ret;
+
+ /* Write capture setting */
+ ret = ov5640_load_regs(sensor, mode);
+ if (ret < 0)
+ return ret;
+
+ /* read capture VTS */
+ ret = ov5640_get_vts(sensor);
+ ret = ov5640_get_hts(sensor);
+ ret = ov5640_get_sysclk(sensor);
+
+ /* calculate capture banding filter */
+ ret = ov5640_get_light_freq(sensor);
+
+ if (light_freq == 60) {
+ /* 60Hz */
+ cap_bandfilt = cap_sysclk * 100 / cap_hts * 100 / 120;
+ } else {
+ /* 50Hz */
+ cap_bandfilt = cap_sysclk * 100 / cap_hts;
+ }
+ cap_maxband = (int)((cap_vts - 4) / cap_bandfilt);
+
+ /* calculate capture shutter/gain16 */
+ if (average > sensor->ae_low && average < sensor->ae_high) {
+ /* in stable range */
+ cap_gain16_shutter =
+ prev_gain16 * prev_shutter *
+ cap_sysclk / sensor->prev_sysclk *
+ sensor->prev_hts / cap_hts *
+ sensor->ae_target / average;
+ } else {
+ cap_gain16_shutter =
+ prev_gain16 * prev_shutter *
+ cap_sysclk / sensor->prev_sysclk *
+ sensor->prev_hts / cap_hts;
+ }
+
+ /* gain to shutter */
+ if (cap_gain16_shutter < (cap_bandfilt * 16)) {
+ /* shutter < 1/100 */
+ cap_shutter = cap_gain16_shutter / 16;
+ if (cap_shutter < 1)
+ cap_shutter = 1;
+
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ if (cap_gain16 < 16)
+ cap_gain16 = 16;
+ } else {
+ if (cap_gain16_shutter > (cap_bandfilt * cap_maxband * 16)) {
+ /* exposure reach max */
+ cap_shutter = cap_bandfilt * cap_maxband;
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ } else {
+ /* 1/100 < (cap_shutter = n/100) =< max */
+ cap_shutter =
+ ((int)(cap_gain16_shutter / 16 / cap_bandfilt))
+ * cap_bandfilt;
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ }
+ }
+
+ /* write capture gain */
+ ret = ov5640_set_gain(sensor, cap_gain16);
+ if (ret < 0)
+ return ret;
+
+ /* write capture shutter */
+ if (cap_shutter > (cap_vts - 4)) {
+ cap_vts = cap_shutter + 4;
+ ret = ov5640_set_vts(sensor, cap_vts);
+ if (ret < 0)
+}
+
+/*
+ * if sensor changes inside scaling or subsampling
+ * change mode directly
+ */
+static int ov5640_set_mode_direct(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *mode)
+{
+ int ret;
+
+ if (mode->reg_data == NULL)
+ return -EINVAL;
+
+ /* turn off AE/AG */
+ ret = ov5640_set_agc_aec(sensor, false);
+ if (ret < 0)
+ return ret;
+
+ /* Write capture setting */
+ ret = ov5640_load_regs(sensor, mode);
+ if (ret < 0)
+ return ret;
+
+ return ov5640_set_agc_aec(sensor, true);
+}
+
+static int ov5640_set_mode(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *orig_mode)
+{
+ const struct ov5640_mode_info *mode = sensor->current_mode;
+ enum ov5640_downsize_mode dn_mode, orig_dn_mode;
+ int ret;
+
+ dn_mode = mode->dn_mode;
+ orig_dn_mode = orig_mode->dn_mode;
+
+ if ((dn_mode == SUBSAMPLING && orig_dn_mode == SCALING) ||
+ (dn_mode == SCALING && orig_dn_mode == SUBSAMPLING)) {
+ /*
+ * change between subsampling and scaling
+ * go through exposure calucation
+ */
+ ret = ov5640_set_mode_exposure_calc(sensor, mode);
+ } else {
+ /*
+ * change inside subsampling or scaling
+ * download firmware directly
+ */
+ ret = ov5640_set_mode_direct(sensor, mode);
+ }
+
+ if (ret < 0)
+ return ret;
+
+ ret = ov5640_set_ae_target(sensor, sensor->ae_target);
+ if (ret < 0)
+ return ret;
+
+ sensor->pending_mode_change = false;
+
+static int ov5640_restore_mode(struct ov5640_dev *sensor)
+{
+ int ret;
+
+ /* first load the initial register values */
+ ret = ov5640_load_regs(sensor, &ov5640_mode_init_data);
+ if (ret < 0)
+ return ret;
+
+ /* now restore the last capture mode */
+ return ov5640_set_mode(sensor, &ov5640_mode_init_data);
+}
+
+static void ov5640_power(struct ov5640_dev *sensor, bool enable)
+{
+ if (sensor->pwdn_gpio)
+ gpiod_set_value(sensor->pwdn_gpio, enable ? 0 : 1);
+}
+
+static void ov5640_reset(struct ov5640_dev *sensor)
+{
+ if (!sensor->reset_gpio)
+ return;
+
+ gpiod_set_value(sensor->reset_gpio, 0);
+
+ /* camera power cycle */
+ ov5640_power(sensor, false);
+ usleep_range(5000, 10000);
+ ov5640_power(sensor, true);
+ usleep_range(5000, 10000);
+
+ gpiod_set_value(sensor->reset_gpio, 1);
+ usleep_range(1000, 2000);
+
+ gpiod_set_value(sensor->reset_gpio, 0);
+ usleep_range(5000, 10000);
+}
+
+static int ov5640_set_power(struct ov5640_dev *sensor, bool on)
+{
+ clk_prepare_enable(sensor->xclk);
+
+ ret = regulator_bulk_enable(OV5640_NUM_SUPPLIES,
+ sensor->supplies);
+ if (ret)
+ goto xclk_off;
+
+ ov5640_reset(sensor);
+ ov5640_power(sensor, true);
+
+ ret = ov5640_init_slave_id(sensor);
+ if (ret)
+ goto power_off;
+
+ ret = ov5640_restore_mode(sensor);
+ if (ret)
+ goto power_off;
+
+ /*
+ * start streaming briefly followed by stream off in
+ * order to coax the clock lane into LP-11 state.
+ */
+ ov5640_set_stream(sensor, true);
+ usleep_range(1000, 2000);
+ ov5640_set_stream(sensor, false);
+
+ ov5640_power(sensor, false);
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+xclk_off:
+ clk_disable_unprepare(sensor->xclk);
+
+static int ov5640_s_power(struct v4l2_subdev *sd, int on)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ ret = ov5640_set_power(sensor, !!on);
+ if (ret)
+ WARN_ON(sensor->power_count < 0);
+out:
+ mutex_unlock(&sensor->lock);
+
+ if (on && !ret && sensor->power_count == 1) {
+ /* restore controls */
+ ret = v4l2_ctrl_handler_setup(&sensor->ctrls.handler);
+ }
+ struct v4l2_fract *fi,
+ u32 width, u32 height)
+{
+ const struct ov5640_mode_info *mode;
+ u32 minfps, maxfps, fps;
+ int ret;
+
+ minfps = ov5640_framerates[OV5640_15_FPS];
+ maxfps = ov5640_framerates[OV5640_30_FPS];
+
+ if (fi->numerator == 0) {
+ fi->denominator = maxfps;
+ fi->numerator = 1;
+ return OV5640_30_FPS;
+ }
+
+ fps = DIV_ROUND_CLOSEST(fi->denominator, fi->numerator);
+
+ fi->numerator = 1;
+ if (fps > maxfps)
+ fi->denominator = maxfps;
+ else if (fps < minfps)
+ fi->denominator = minfps;
+ else if (2 * fps >= 2 * minfps + (maxfps - minfps))
+ fi->denominator = maxfps;
+ else
+ fi->denominator = minfps;
+
+ ret = (fi->denominator == minfps) ? OV5640_15_FPS : OV5640_30_FPS;
+
+ mode = ov5640_find_mode(sensor, ret, width, height, false);
+ return mode ? ret : -EINVAL;
+}
+
+static int ov5640_g_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_captureparm *cparm = &a->parm.capture;
+
+ mutex_lock(&sensor->lock);
+
+ /* This is the only case currently handled. */
+ memset(a, 0, sizeof(*a));
+ a->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ cparm->capability = sensor->streamcap.capability;
+ cparm->timeperframe = sensor->streamcap.timeperframe;
+ cparm->capturemode = sensor->streamcap.capturemode;
+
+ mutex_unlock(&sensor->lock);
+
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_fract *tpf = &a->parm.capture.timeperframe;
+ const struct ov5640_mode_info *mode;
+ int frame_rate, ret = 0;
+
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+
+ frame_rate = ov5640_try_frame_interval(sensor, tpf,
+ mode->width, mode->height);
+ if (frame_rate < 0) {
+ ret = frame_rate;
+ sensor->streamcap.timeperframe = *tpf;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ fmt = v4l2_subdev_get_try_format(&sensor->sd, cfg,
+ format->pad);
+ else
+ fmt = &sensor->fmt;
+
+ format->format = *fmt;
+
+ mutex_unlock(&sensor->lock);
+
+ struct v4l2_mbus_framefmt *fmt,
+ enum ov5640_frame_rate fr,
+ const struct ov5640_mode_info **new_mode)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ const struct ov5640_mode_info *mode;
+
+ mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
+ if (!mode)
+ return -EINVAL;
+
+ fmt->width = mode->width;
+ fmt->height = mode->height;
+ fmt->code = sensor->fmt.code;
+
+ if (new_mode)
+ *new_mode = mode;
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ const struct ov5640_mode_info *new_mode;
+ int ret;
+
+ if (format->pad != 0)
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+ sensor->current_fr, &new_mode);
+ if (ret)
+ goto out;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ struct v4l2_mbus_framefmt *fmt =
+ v4l2_subdev_get_try_format(sd, cfg, 0);
+
+ *fmt = format->format;
+ sensor->fmt = format->format;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+/*
+ * Sensor Controls.
+ */
+
+static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ mutex_lock(&sensor->lock);
+ if (!ctrl->val)
+ goto out;
+ sensor->ctrls.gain->val = ov5640_get_gain(sensor);
+ break;
+ case V4L2_CID_EXPOSURE_AUTO:
+ if (ctrl->val == V4L2_EXPOSURE_MANUAL)
+ goto out;
+ sensor->ctrls.exposure->val = ov5640_get_exposure(sensor);
+ break;
+ }
+
+out:
+ mutex_unlock(&sensor->lock);
+{
+ struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ /*
+ * If the device is not powered up by the host driver do
+ * not apply any controls to H/W at this time. Instead
+ * the controls will be restored right after power-up.
+ */
+ if (sensor->power_count == 0)
+ goto out;
+
+ switch (ctrl->id) {
+ case V4L2_CID_AUTOGAIN:
+ ret = ov5640_set_gain(sensor, ctrl->val);
+ break;
+ case V4L2_CID_EXPOSURE_AUTO:
+ ret = ov5640_set_exposure(sensor, ctrl->val);
+ break;
+ case V4L2_CID_AUTO_WHITE_BALANCE:
+ ret = ov5640_set_white_balance(sensor, ctrl->val);
+ break;
+ case V4L2_CID_HUE:
+ ret = ov5640_set_hue(sensor, ctrl->val);
+ break;
+ case V4L2_CID_CONTRAST:
+ ret = ov5640_set_contrast(sensor, ctrl->val);
+ break;
+ case V4L2_CID_SATURATION:
+ ret = ov5640_set_saturation(sensor, ctrl->val);
+ break;
+ case V4L2_CID_TEST_PATTERN:
+ ret = ov5640_set_test_pattern(sensor, ctrl->val);
+ }
+
+out:
+ mutex_unlock(&sensor->lock);
+ .g_volatile_ctrl = ov5640_g_volatile_ctrl,
+ .s_ctrl = ov5640_s_ctrl,
+};
+
+static const char * const test_pattern_menu[] = {
+ "Disabled",
+ "Color bars",
+};
+
+static int ov5640_init_controls(struct ov5640_dev *sensor)
+{
+ const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+ struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+ int ret;
+
+ v4l2_ctrl_handler_init(hdl, 32);
+
+ /* Auto/manual white balance */
+ ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
+ V4L2_CID_AUTO_WHITE_BALANCE,
+ 0, 1, 1, 1);
+ ctrls->blue_balance = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BLUE_BALANCE,
+ 0, 4095, 1, 0);
+ ctrls->red_balance = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_RED_BALANCE,
+ 0, 4095, 1, 0);
+ /* Auto/manual exposure */
+ ctrls->auto_exp = v4l2_ctrl_new_std_menu(hdl, ops,
+ V4L2_CID_EXPOSURE_AUTO,
+ V4L2_EXPOSURE_MANUAL, 0,
+ V4L2_EXPOSURE_AUTO);
+ ctrls->exposure = v4l2_ctrl_new_std(hdl, ops,
+ V4L2_CID_EXPOSURE_ABSOLUTE,
+ 0, 65535, 1, 0);
+ /* Auto/manual gain */
+ ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN,
+ 0, 1, 1, 1);
+ ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
+ 0, 1023, 1, 0);
+
+ ctrls->saturation = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_SATURATION,
+ 0, 255, 1, 64);
+ ctrls->hue = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HUE,
+ 0, 359, 1, 0);
+ ctrls->contrast = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_CONTRAST,
+ 0, 255, 1, 0);
+ ctrls->test_pattern =
+ v4l2_ctrl_new_std_menu_items(hdl, ops, V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(test_pattern_menu) - 1,
+ 0, 0, test_pattern_menu);
+
+ if (hdl->error) {
+ ret = hdl->error;
+ goto free_ctrls;
+ }
+
+ ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
+ ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ v4l2_ctrl_auto_cluster(3, &ctrls->auto_wb, 0, false);
+ v4l2_ctrl_auto_cluster(2, &ctrls->auto_gain, 0, true);
+ v4l2_ctrl_auto_cluster(2, &ctrls->auto_exp, 1, true);
+
+ sensor->sd.ctrl_handler = hdl;
+ return 0;
+
+free_ctrls:
+ v4l2_ctrl_handler_free(hdl);
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->pad != 0)
+ return -EINVAL;
+ if (fse->index >= OV5640_NUM_MODES)
+ return -EINVAL;
+
+ fse->min_width = fse->max_width =
+ ov5640_mode_data[0][fse->index].width;
+ fse->min_height = fse->max_height =
+ ov5640_mode_data[0][fse->index].height;
+
+ struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_interval_enum *fie)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_fract tpf;
+ int ret;
+
+ if (fie->pad != 0)
+ return -EINVAL;
+ if (fie->index >= OV5640_NUM_FRAMERATES)
+ return -EINVAL;
+
+ tpf.numerator = 1;
+ tpf.denominator = ov5640_framerates[fie->index];
+
+ ret = ov5640_try_frame_interval(sensor, &tpf,
+ fie->width, fie->height);
+ if (ret < 0)
+ return -EINVAL;
+
+ fie->interval = tpf;
+
+ mutex_lock(&sensor->lock);
+ fi->interval = sensor->streamcap.timeperframe;
+ mutex_unlock(&sensor->lock);
+
+ const struct ov5640_mode_info *mode;
+ int frame_rate, ret = 0;
+
+ if (fi->pad != 0)
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+
+ frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
+ mode->width, mode->height);
+ if (frame_rate < 0)
+ frame_rate = OV5640_15_FPS;
+
+ sensor->current_fr = frame_rate;
+ sensor->streamcap.timeperframe = fi->interval;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+
+ if (code->pad != 0)
+ return -EINVAL;
+
+ code->code = sensor->fmt.code;
+
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ if (sensor->streaming == !enable) {
+ if (enable && sensor->pending_mode_change) {
+ ret = ov5640_set_mode(sensor, sensor->current_mode);
+ if (ret)
+ if (!ret)
+ sensor->streaming = enable;
+ }
+out:
+ mutex_unlock(&sensor->lock);
+ .s_power = ov5640_s_power,
+};
+
+static const struct v4l2_subdev_video_ops ov5640_video_ops = {
+ .s_parm = ov5640_s_parm,
+ .g_parm = ov5640_g_parm,
+ .g_frame_interval = ov5640_g_frame_interval,
+ .s_frame_interval = ov5640_s_frame_interval,
+ .s_stream = ov5640_s_stream,
+};
+
+static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
+ .enum_mbus_code = ov5640_enum_mbus_code,
+ .get_fmt = ov5640_get_fmt,
+ .set_fmt = ov5640_set_fmt,
+ .enum_frame_size = ov5640_enum_frame_size,
+ .enum_frame_interval = ov5640_enum_frame_interval,
+};
+
+static const struct v4l2_subdev_ops ov5640_subdev_ops = {
+ .core = &ov5640_core_ops,
+ .video = &ov5640_video_ops,
+ .pad = &ov5640_pad_ops,
+};
+
+static int ov5640_get_regulators(struct ov5640_dev *sensor)
+{
+ int i;
+
+ for (i = 0; i < OV5640_NUM_SUPPLIES; i++)
+ sensor->supplies[i].supply = ov5640_supply_name[i];
+
+ return devm_regulator_bulk_get(&sensor->i2c_client->dev,
+ OV5640_NUM_SUPPLIES,
+ sensor->supplies);
+}
+
+static int ov5640_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &client->dev;
+ struct device_node *endpoint;
+ struct ov5640_dev *sensor;
+ int ret;
+
+ sensor = devm_kzalloc(dev, sizeof(*sensor), GFP_KERNEL);
+ if (!sensor)
+ return -ENOMEM;
+
+ sensor->i2c_client = client;
+ sensor->fmt.code = MEDIA_BUS_FMT_UYVY8_2X8;
+ sensor->fmt.width = 640;
+ sensor->fmt.height = 480;
+ sensor->fmt.field = V4L2_FIELD_NONE;
+ sensor->streamcap.capability = V4L2_CAP_TIMEPERFRAME;
+ sensor->streamcap.capturemode = 0;
+ sensor->streamcap.timeperframe.denominator =
+ ov5640_framerates[OV5640_30_FPS];
+ sensor->streamcap.timeperframe.numerator = 1;
+ sensor->current_fr = OV5640_30_FPS;
+ sensor->current_mode =
+ &ov5640_mode_data[OV5640_30_FPS][OV5640_MODE_VGA_640_480];
+ sensor->pending_mode_change = true;
+
+ sensor->ae_target = 52;
+
+ endpoint = of_graph_get_next_endpoint(client->dev.of_node, NULL);
+ if (!endpoint) {
+ dev_err(dev, "endpoint node not found\n");
+ of_node_put(endpoint);
+
+ if (sensor->ep.bus_type != V4L2_MBUS_CSI2) {
+ dev_err(dev, "invalid bus type, must be MIPI CSI2\n");
+ sensor->xclk = devm_clk_get(dev, "xclk");
+ if (IS_ERR(sensor->xclk)) {
+ dev_err(dev, "failed to get xclk\n");
+ return PTR_ERR(sensor->xclk);
+ }
+
+ sensor->xclk_freq = clk_get_rate(sensor->xclk);
+ if (sensor->xclk_freq < OV5640_XCLK_MIN ||
+ sensor->xclk_freq > OV5640_XCLK_MAX) {
+ dev_err(dev, "xclk frequency out of range: %d Hz\n",
+ sensor->xclk_freq);
+ sensor->pwdn_gpio = devm_gpiod_get_optional(dev, "powerdown",
+ GPIOD_OUT_HIGH);
+ /* request optional reset pin */
+ sensor->reset_gpio = devm_gpiod_get_optional(dev, "reset",
+ GPIOD_OUT_HIGH);
+
+ v4l2_i2c_subdev_init(&sensor->sd, client, &ov5640_subdev_ops);
+
+ sensor->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
+ sensor->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
+ ret = media_entity_pads_init(&sensor->sd.entity, 1, &sensor->pad);
+
+ ret = ov5640_init_controls(sensor);
+ if (ret)
+ goto entity_cleanup;
+
+ ret = v4l2_async_register_subdev(&sensor->sd);
+ if (ret)
+ goto free_ctrls;
+ v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+entity_cleanup:
+ mutex_destroy(&sensor->lock);
+ media_entity_cleanup(&sensor->sd.entity);
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+{
+ struct v4l2_subdev *sd = i2c_get_clientdata(client);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+
+ v4l2_async_unregister_subdev(&sensor->sd);
+ mutex_destroy(&sensor->lock);
+ media_entity_cleanup(&sensor->sd.entity);
+ v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+
+ {"ov5640", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, ov5640_id);
+
+static const struct of_device_id ov5640_dt_ids[] = {
+ { .compatible = "ovti,ov5640" },
+
+static struct i2c_driver ov5640_i2c_driver = {
+ .driver = {
+ .name = "ov5640",
+ .of_match_table = ov5640_dt_ids,
+ },
+ .id_table = ov5640_id,
+ .probe = ov5640_probe,
+ .remove = ov5640_remove,
+};
+
+module_i2c_driver(ov5640_i2c_driver);
+
+MODULE_DESCRIPTION("OV5640 MIPI Camera Subdev Driver");
branch, modified heavily to bring forward to latest interfaces and code
cleanup.
drivers/media/i2c/Kconfig | 7 +
drivers/media/i2c/Makefile | 1 +
drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 2239 insertions(+)
create mode 100644 drivers/media/i2c/ov5640.c
diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index cee1dae..bf67661 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -531,6 +531,13 @@ config VIDEO_OV2659
To compile this driver as a module, choose M here: the
module will be called ov2659.
+config VIDEO_OV5640
+ tristate "OmniVision OV5640 sensor support"
+ depends on GPIOLIB && VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API
+ ---help---
+ This is a V4L2 sensor-level driver for the Omnivision
+ OV5640 camera sensor with a MIPI CSI-2 interface.
+
config VIDEO_OV7640
tristate "OmniVision OV7640 sensor support"
depends on I2C && VIDEO_V4L2
diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
index 5bc7bbe..3a9d73a 100644
--- a/drivers/media/i2c/Makefile
+++ b/drivers/media/i2c/Makefile
@@ -57,6 +57,7 @@ obj-$(CONFIG_VIDEO_VP27SMPX) += vp27smpx.o
obj-$(CONFIG_VIDEO_SONY_BTF_MPX) += sony-btf-mpx.o
obj-$(CONFIG_VIDEO_UPD64031A) += upd64031a.o
obj-$(CONFIG_VIDEO_UPD64083) += upd64083.o
+obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
obj-$(CONFIG_VIDEO_OV7670) += ov7670.o
obj-$(CONFIG_VIDEO_OV9650) += ov9650.o
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
new file mode 100644
index 0000000..2d78ec8
--- /dev/null
+++ b/drivers/media/i2c/ov5640.c
@@ -0,0 +1,2231 @@
+/*
+ * Copyright (C) 2011-2013 Freescale Semiconductor, Inc. All Rights Reserved.
+ * Copyright (C) 2014-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+#include <linux/ctype.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/i2c.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/gpio/consumer.h>
+#include <linux/regulator/consumer.h>
+#include <media/v4l2-async.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-of.h>
+#include <media/v4l2-subdev.h>
+
+/* min/typical/max system clock (xclk) frequencies */
+#define OV5640_XCLK_MIN 6000000
+#define OV5640_XCLK_MAX 24000000
+
+/*
+ * FIXME: there is no subdev API to set the MIPI CSI-2
+ * virtual channel yet, so this is hardcoded for now.
+ */
+#define OV5640_MIPI_VC 1
+
+#define OV5640_DEFAULT_SLAVE_ID 0x3c
+
+#define OV5640_REG_CHIP_ID 0x300a
+#define OV5640_REG_PAD_OUTPUT00 0x3019
+#define OV5640_REG_SC_PLL_CTRL0 0x3034
+#define OV5640_REG_SC_PLL_CTRL1 0x3035
+#define OV5640_REG_SC_PLL_CTRL2 0x3036
+#define OV5640_REG_SC_PLL_CTRL3 0x3037
+#define OV5640_REG_SLAVE_ID 0x3100
+#define OV5640_REG_SYS_ROOT_DIVIDER 0x3108
+#define OV5640_REG_AWB_R_GAIN 0x3400
+#define OV5640_REG_AWB_G_GAIN 0x3402
+#define OV5640_REG_AWB_B_GAIN 0x3404
+#define OV5640_REG_AWB_MANUAL_CTRL 0x3406
+#define OV5640_REG_AEC_PK_EXPOSURE_HI 0x3500
+#define OV5640_REG_AEC_PK_EXPOSURE_MED 0x3501
+#define OV5640_REG_AEC_PK_EXPOSURE_LO 0x3502
+#define OV5640_REG_AEC_PK_MANUAL 0x3503
+#define OV5640_REG_AEC_PK_REAL_GAIN 0x350a
+#define OV5640_REG_AEC_PK_VTS 0x350c
+#define OV5640_REG_TIMING_HTS 0x380c
+#define OV5640_REG_TIMING_VTS 0x380e
+#define OV5640_REG_TIMING_TC_REG21 0x3821
+#define OV5640_REG_AEC_CTRL00 0x3a00
+#define OV5640_REG_AEC_B50_STEP 0x3a08
+#define OV5640_REG_AEC_B60_STEP 0x3a0a
+#define OV5640_REG_AEC_CTRL0D 0x3a0d
+#define OV5640_REG_AEC_CTRL0E 0x3a0e
+#define OV5640_REG_AEC_CTRL0F 0x3a0f
+#define OV5640_REG_AEC_CTRL10 0x3a10
+#define OV5640_REG_AEC_CTRL11 0x3a11
+#define OV5640_REG_AEC_CTRL1B 0x3a1b
+#define OV5640_REG_AEC_CTRL1E 0x3a1e
+#define OV5640_REG_AEC_CTRL1F 0x3a1f
+#define OV5640_REG_HZ5060_CTRL00 0x3c00
+#define OV5640_REG_HZ5060_CTRL01 0x3c01
+#define OV5640_REG_SIGMADELTA_CTRL0C 0x3c0c
+#define OV5640_REG_FRAME_CTRL01 0x4202
+#define OV5640_REG_MIPI_CTRL00 0x4800
+#define OV5640_REG_DEBUG_MODE 0x4814
+#define OV5640_REG_PRE_ISP_TEST_SET1 0x503d
+#define OV5640_REG_SDE_CTRL0 0x5580
+#define OV5640_REG_SDE_CTRL1 0x5581
+#define OV5640_REG_SDE_CTRL3 0x5583
+#define OV5640_REG_SDE_CTRL4 0x5584
+#define OV5640_REG_SDE_CTRL5 0x5585
+#define OV5640_REG_AVG_READOUT 0x56a1
+
+enum ov5640_mode_id {
+ OV5640_MODE_QCIF_176_144 = 0,
+ OV5640_MODE_QVGA_320_240,
+ OV5640_MODE_VGA_640_480,
+ OV5640_MODE_NTSC_720_480,
+ OV5640_MODE_PAL_720_576,
+ OV5640_MODE_XGA_1024_768,
+ OV5640_MODE_720P_1280_720,
+ OV5640_MODE_1080P_1920_1080,
+ OV5640_MODE_QSXGA_2592_1944,
+ OV5640_NUM_MODES,
+};
+
+enum ov5640_frame_rate {
+ OV5640_15_FPS = 0,
+ OV5640_30_FPS,
+ OV5640_NUM_FRAMERATES,
+};
+
+static const int ov5640_framerates[] = {
+ [OV5640_15_FPS] = 15,
+ [OV5640_30_FPS] = 30,
+};
+
+/* regulator supplies */
+static const char * const ov5640_supply_name[] = {
+ "DOVDD", /* Digital I/O (1.8V) suppply */
+ "DVDD", /* Digital Core (1.5V) supply */
+ "AVDD", /* Analog (2.8V) supply */
+};
+
+#define OV5640_NUM_SUPPLIES ARRAY_SIZE(ov5640_supply_name)
+
+/*
+ * image size under 1280 * 960 are SUBSAMPLING
+ * image size upper 1280 * 960 are SCALING
+ */
+enum ov5640_downsize_mode {
+ SUBSAMPLING,
+ SCALING,
+};
+
+struct reg_value {
+ u16 reg_addr;
+ u8 val;
+ u8 mask;
+ u32 delay_ms;
+};
+
+struct ov5640_mode_info {
+ enum ov5640_mode_id id;
+ enum ov5640_downsize_mode dn_mode;
+ u32 width;
+ u32 height;
+ const struct reg_value *reg_data;
+ u32 reg_data_size;
+};
+
+struct ov5640_ctrls {
+ struct v4l2_ctrl_handler handler;
+ struct {
+ struct v4l2_ctrl *auto_exp;
+ struct v4l2_ctrl *exposure;
+ };
+ struct {
+ struct v4l2_ctrl *auto_wb;
+ struct v4l2_ctrl *blue_balance;
+ struct v4l2_ctrl *red_balance;
+ };
+ struct {
+ struct v4l2_ctrl *auto_gain;
+ struct v4l2_ctrl *gain;
+ };
+ struct v4l2_ctrl *brightness;
+ struct v4l2_ctrl *saturation;
+ struct v4l2_ctrl *contrast;
+ struct v4l2_ctrl *hue;
+ struct v4l2_ctrl *test_pattern;
+};
+
+struct ov5640_dev {
+ struct i2c_client *i2c_client;
+ struct v4l2_subdev sd;
+ struct media_pad pad;
+ struct v4l2_of_endpoint ep; /* the parsed DT endpoint info */
+ struct clk *xclk; /* system clock to OV5640 */
+ u32 xclk_freq;
+
+ struct regulator_bulk_data supplies[OV5640_NUM_SUPPLIES];
+ struct gpio_desc *reset_gpio;
+ struct gpio_desc *pwdn_gpio;
+
+ /* lock to protect all members below */
+ struct mutex lock;
+
+ int power_count;
+ /* lock to protect all members below */
+ struct mutex lock;
+
+
+ struct v4l2_mbus_framefmt fmt;
+ struct v4l2_captureparm streamcap;
+
+ const struct ov5640_mode_info *current_mode;
+ enum ov5640_frame_rate current_fr;
+
+ struct ov5640_ctrls ctrls;
+
+ u32 prev_sysclk, prev_hts;
+ u32 ae_low, ae_high, ae_target;
+
+ bool pending_mode_change;
+ bool streaming;
+};
+
+static inline struct ov5640_dev *to_ov5640_dev(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct ov5640_dev, sd);
+}
+
+static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
+{
+ return &container_of(ctrl->handler, struct ov5640_dev,
+ ctrls.handler)->sd;
+}
+
+/*
+ * FIXME: all of these register tables are likely filled with
+ * entries that set the register to their power-on default values,
+ * and which are otherwise not touched by this driver. Those entries
+ * should be identified and removed to speed register load time
+ * over i2c.
+ */
+
+static const struct reg_value ov5640_init_setting_30fps_VGA[] = {
+
+ {0x3103, 0x11, 0, 0}, {0x3008, 0x82, 0, 5}, {0x3008, 0x42, 0, 0},
+ {0x3103, 0x03, 0, 0}, {0x3017, 0x00, 0, 0}, {0x3018, 0x00, 0, 0},
+ {0x3034, 0x18, 0, 0}, {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0},
+ {0x3037, 0x13, 0, 0}, {0x3108, 0x01, 0, 0}, {0x3630, 0x36, 0, 0},
+ {0x3631, 0x0e, 0, 0}, {0x3632, 0xe2, 0, 0}, {0x3633, 0x12, 0, 0},
+ {0x3621, 0xe0, 0, 0}, {0x3704, 0xa0, 0, 0}, {0x3703, 0x5a, 0, 0},
+ {0x3715, 0x78, 0, 0}, {0x3717, 0x01, 0, 0}, {0x370b, 0x60, 0, 0},
+ {0x3705, 0x1a, 0, 0}, {0x3905, 0x02, 0, 0}, {0x3906, 0x10, 0, 0},
+ {0x3901, 0x0a, 0, 0}, {0x3731, 0x12, 0, 0}, {0x3600, 0x08, 0, 0},
+ {0x3601, 0x33, 0, 0}, {0x302d, 0x60, 0, 0}, {0x3620, 0x52, 0, 0},
+ {0x371b, 0x20, 0, 0}, {0x471c, 0x50, 0, 0}, {0x3a13, 0x43, 0, 0},
+ {0x3a18, 0x00, 0, 0}, {0x3a19, 0xf8, 0, 0}, {0x3635, 0x13, 0, 0},
+ {0x3636, 0x03, 0, 0}, {0x3634, 0x40, 0, 0}, {0x3622, 0x01, 0, 0},
+ {0x3c01, 0xa4, 0, 0}, {0x3c04, 0x28, 0, 0}, {0x3c05, 0x98, 0, 0},
+ {0x3c06, 0x00, 0, 0}, {0x3c07, 0x08, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x3000, 0x00, 0, 0},
+ {0x3002, 0x1c, 0, 0}, {0x3004, 0xff, 0, 0}, {0x3006, 0xc3, 0, 0},
+ {0x300e, 0x45, 0, 0}, {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
+ {0x501f, 0x00, 0, 0}, {0x4713, 0x03, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x440e, 0x00, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x4837, 0x0a, 0, 0}, {0x4800, 0x04, 0, 0}, {0x3824, 0x02, 0, 0},
+ {0x5000, 0xa7, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x5180, 0xff, 0, 0},
+ {0x5181, 0xf2, 0, 0}, {0x5182, 0x00, 0, 0}, {0x5183, 0x14, 0, 0},
+ {0x5184, 0x25, 0, 0}, {0x5185, 0x24, 0, 0}, {0x5186, 0x09, 0, 0},
+ {0x5187, 0x09, 0, 0}, {0x5188, 0x09, 0, 0}, {0x5189, 0x88, 0, 0},
+ {0x518a, 0x54, 0, 0}, {0x518b, 0xee, 0, 0}, {0x518c, 0xb2, 0, 0},
+ {0x518d, 0x50, 0, 0}, {0x518e, 0x34, 0, 0}, {0x518f, 0x6b, 0, 0},
+ {0x5190, 0x46, 0, 0}, {0x5191, 0xf8, 0, 0}, {0x5192, 0x04, 0, 0},
+ {0x5193, 0x70, 0, 0}, {0x5194, 0xf0, 0, 0}, {0x5195, 0xf0, 0, 0},
+ {0x5196, 0x03, 0, 0}, {0x5197, 0x01, 0, 0}, {0x5198, 0x04, 0, 0},
+ {0x5199, 0x6c, 0, 0}, {0x519a, 0x04, 0, 0}, {0x519b, 0x00, 0, 0},
+ {0x519c, 0x09, 0, 0}, {0x519d, 0x2b, 0, 0}, {0x519e, 0x38, 0, 0},
+ {0x5381, 0x1e, 0, 0}, {0x5382, 0x5b, 0, 0}, {0x5383, 0x08, 0, 0},
+ {0x5384, 0x0a, 0, 0}, {0x5385, 0x7e, 0, 0}, {0x5386, 0x88, 0, 0},
+ {0x5387, 0x7c, 0, 0}, {0x5388, 0x6c, 0, 0}, {0x5389, 0x10, 0, 0},
+ {0x538a, 0x01, 0, 0}, {0x538b, 0x98, 0, 0}, {0x5300, 0x08, 0, 0},
+ {0x5301, 0x30, 0, 0}, {0x5302, 0x10, 0, 0}, {0x5303, 0x00, 0, 0},
+ {0x5304, 0x08, 0, 0}, {0x5305, 0x30, 0, 0}, {0x5306, 0x08, 0, 0},
+ {0x5307, 0x16, 0, 0}, {0x5309, 0x08, 0, 0}, {0x530a, 0x30, 0, 0},
+ {0x530b, 0x04, 0, 0}, {0x530c, 0x06, 0, 0}, {0x5480, 0x01, 0, 0},
+ {0x5481, 0x08, 0, 0}, {0x5482, 0x14, 0, 0}, {0x5483, 0x28, 0, 0},
+ {0x5484, 0x51, 0, 0}, {0x5485, 0x65, 0, 0}, {0x5486, 0x71, 0, 0},
+ {0x5487, 0x7d, 0, 0}, {0x5488, 0x87, 0, 0}, {0x5489, 0x91, 0, 0},
+ {0x548a, 0x9a, 0, 0}, {0x548b, 0xaa, 0, 0}, {0x548c, 0xb8, 0, 0},
+ {0x548d, 0xcd, 0, 0}, {0x548e, 0xdd, 0, 0}, {0x548f, 0xea, 0, 0},
+ {0x5490, 0x1d, 0, 0}, {0x5580, 0x02, 0, 0}, {0x5583, 0x40, 0, 0},
+ {0x5584, 0x10, 0, 0}, {0x5589, 0x10, 0, 0}, {0x558a, 0x00, 0, 0},
+ {0x558b, 0xf8, 0, 0}, {0x5800, 0x23, 0, 0}, {0x5801, 0x14, 0, 0},
+ {0x5802, 0x0f, 0, 0}, {0x5803, 0x0f, 0, 0}, {0x5804, 0x12, 0, 0},
+ {0x5805, 0x26, 0, 0}, {0x5806, 0x0c, 0, 0}, {0x5807, 0x08, 0, 0},
+ {0x5808, 0x05, 0, 0}, {0x5809, 0x05, 0, 0}, {0x580a, 0x08, 0, 0},
+ {0x580b, 0x0d, 0, 0}, {0x580c, 0x08, 0, 0}, {0x580d, 0x03, 0, 0},
+ {0x580e, 0x00, 0, 0}, {0x580f, 0x00, 0, 0}, {0x5810, 0x03, 0, 0},
+ {0x5811, 0x09, 0, 0}, {0x5812, 0x07, 0, 0}, {0x5813, 0x03, 0, 0},
+ {0x5814, 0x00, 0, 0}, {0x5815, 0x01, 0, 0}, {0x5816, 0x03, 0, 0},
+ {0x5817, 0x08, 0, 0}, {0x5818, 0x0d, 0, 0}, {0x5819, 0x08, 0, 0},
+ {0x581a, 0x05, 0, 0}, {0x581b, 0x06, 0, 0}, {0x581c, 0x08, 0, 0},
+ {0x581d, 0x0e, 0, 0}, {0x581e, 0x29, 0, 0}, {0x581f, 0x17, 0, 0},
+ {0x5820, 0x11, 0, 0}, {0x5821, 0x11, 0, 0}, {0x5822, 0x15, 0, 0},
+ {0x5823, 0x28, 0, 0}, {0x5824, 0x46, 0, 0}, {0x5825, 0x26, 0, 0},
+ {0x5826, 0x08, 0, 0}, {0x5827, 0x26, 0, 0}, {0x5828, 0x64, 0, 0},
+ {0x5829, 0x26, 0, 0}, {0x582a, 0x24, 0, 0}, {0x582b, 0x22, 0, 0},
+ {0x582c, 0x24, 0, 0}, {0x582d, 0x24, 0, 0}, {0x582e, 0x06, 0, 0},
+ {0x582f, 0x22, 0, 0}, {0x5830, 0x40, 0, 0}, {0x5831, 0x42, 0, 0},
+ {0x5832, 0x24, 0, 0}, {0x5833, 0x26, 0, 0}, {0x5834, 0x24, 0, 0},
+ {0x5835, 0x22, 0, 0}, {0x5836, 0x22, 0, 0}, {0x5837, 0x26, 0, 0},
+ {0x5838, 0x44, 0, 0}, {0x5839, 0x24, 0, 0}, {0x583a, 0x26, 0, 0},
+ {0x583b, 0x28, 0, 0}, {0x583c, 0x42, 0, 0}, {0x583d, 0xce, 0, 0},
+ {0x5025, 0x00, 0, 0}, {0x3a0f, 0x30, 0, 0}, {0x3a10, 0x28, 0, 0},
+ {0x3a1b, 0x30, 0, 0}, {0x3a1e, 0x26, 0, 0}, {0x3a11, 0x60, 0, 0},
+ {0x3a1f, 0x14, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3c00, 0x04, 0, 300},
+};
+
+static const struct reg_value ov5640_setting_30fps_VGA_640_480[] = {
+
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x04, 0, 0}, {0x380f, 0x38, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x0e, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3503, 0x00, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_VGA_640_480[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_XGA_1024_768[] = {
+
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x04, 0, 0}, {0x380f, 0x38, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x0e, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3503, 0x00, 0, 0},
+ {0x3808, 0x04, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x03, 0, 0},
+ {0x380b, 0x00, 0, 0}, {0x3035, 0x12, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_XGA_1024_768[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0x80, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x3808, 0x04, 0, 0},
+ {0x3809, 0x00, 0, 0}, {0x380a, 0x03, 0, 0}, {0x380b, 0x00, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_QVGA_320_240[] = {
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x01, 0, 0}, {0x3809, 0x40, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0xf0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_QVGA_320_240[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x01, 0, 0}, {0x3809, 0x40, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0xf0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_QCIF_176_144[] = {
+ {0x3035, 0x14, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x00, 0, 0}, {0x3809, 0xb0, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0x90, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+static const struct reg_value ov5640_setting_15fps_QCIF_176_144[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x00, 0, 0}, {0x3809, 0xb0, 0, 0}, {0x380a, 0x00, 0, 0},
+ {0x380b, 0x90, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_NTSC_720_480[] = {
+ {0x3035, 0x12, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_NTSC_720_480[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x01, 0, 0},
+ {0x380b, 0xe0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x3c, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_PAL_720_576[] = {
+ {0x3035, 0x12, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0x40, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_PAL_720_576[] = {
+ {0x3035, 0x22, 0, 0}, {0x3036, 0x38, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x04, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9b, 0, 0},
+ {0x3808, 0x02, 0, 0}, {0x3809, 0xd0, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0x40, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x68, 0, 0},
+ {0x380e, 0x03, 0, 0}, {0x380f, 0xd8, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x38, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x06, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0xa3, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_720P_1280_720[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
+ {0x3808, 0x05, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0xd0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x64, 0, 0},
+ {0x380e, 0x02, 0, 0}, {0x380f, 0xe4, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
+ {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
+ {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0},
+ {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x02, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0},
+ {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0}, {0x4005, 0x1a, 0, 0},
+ {0x3008, 0x02, 0, 0}, {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_720P_1280_720[] = {
+ {0x3035, 0x41, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x41, 0, 0}, {0x3821, 0x07, 0, 0}, {0x3814, 0x31, 0, 0},
+ {0x3815, 0x31, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0xfa, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x06, 0, 0}, {0x3807, 0xa9, 0, 0},
+ {0x3808, 0x05, 0, 0}, {0x3809, 0x00, 0, 0}, {0x380a, 0x02, 0, 0},
+ {0x380b, 0xd0, 0, 0}, {0x380c, 0x07, 0, 0}, {0x380d, 0x64, 0, 0},
+ {0x380e, 0x02, 0, 0}, {0x380f, 0xe4, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x00, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3709, 0x52, 0, 0}, {0x370c, 0x03, 0, 0}, {0x3a02, 0x02, 0, 0},
+ {0x3a03, 0xe4, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0xbc, 0, 0},
+ {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x72, 0, 0}, {0x3a0e, 0x01, 0, 0},
+ {0x3a0d, 0x02, 0, 0}, {0x3a14, 0x02, 0, 0}, {0x3a15, 0xe4, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x4713, 0x02, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0},
+ {0x3824, 0x04, 0, 0}, {0x5001, 0x83, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_30fps_1080P_1920_1080[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, {0x3035, 0x11, 0, 0},
+ {0x3036, 0x54, 0, 0}, {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
+ {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
+ {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, {0x3808, 0x07, 0, 0},
+ {0x3809, 0x80, 0, 0}, {0x380a, 0x04, 0, 0}, {0x380b, 0x38, 0, 0},
+ {0x380c, 0x09, 0, 0}, {0x380d, 0xc4, 0, 0}, {0x380e, 0x04, 0, 0},
+ {0x380f, 0x60, 0, 0}, {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
+ {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
+ {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0},
+ {0x3a15, 0x60, 0, 0}, {0x4713, 0x02, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0},
+ {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0},
+ {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_1080P_1920_1080[] = {
+ {0x3008, 0x42, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 0}, {0x3035, 0x21, 0, 0},
+ {0x3036, 0x54, 0, 1}, {0x3c07, 0x07, 0, 0}, {0x3c08, 0x00, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3800, 0x01, 0, 0}, {0x3801, 0x50, 0, 0}, {0x3802, 0x01, 0, 0},
+ {0x3803, 0xb2, 0, 0}, {0x3804, 0x08, 0, 0}, {0x3805, 0xef, 0, 0},
+ {0x3806, 0x05, 0, 0}, {0x3807, 0xf1, 0, 0}, {0x3808, 0x07, 0, 0},
+ {0x3809, 0x80, 0, 0}, {0x380a, 0x04, 0, 0}, {0x380b, 0x38, 0, 0},
+ {0x380c, 0x09, 0, 0}, {0x380d, 0xc4, 0, 0}, {0x380e, 0x04, 0, 0},
+ {0x380f, 0x60, 0, 0}, {0x3612, 0x2b, 0, 0}, {0x3708, 0x64, 0, 0},
+ {0x3a02, 0x04, 0, 0}, {0x3a03, 0x60, 0, 0}, {0x3a08, 0x01, 0, 0},
+ {0x3a09, 0x50, 0, 0}, {0x3a0a, 0x01, 0, 0}, {0x3a0b, 0x18, 0, 0},
+ {0x3a0e, 0x03, 0, 0}, {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x04, 0, 0},
+ {0x3a15, 0x60, 0, 0}, {0x4713, 0x02, 0, 0}, {0x4407, 0x04, 0, 0},
+ {0x460b, 0x37, 0, 0}, {0x460c, 0x20, 0, 0}, {0x3824, 0x04, 0, 0},
+ {0x4005, 0x1a, 0, 0}, {0x3008, 0x02, 0, 0}, {0x3503, 0, 0, 0},
+};
+
+static const struct reg_value ov5640_setting_15fps_QSXGA_2592_1944[] = {
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0},
+ {0x3035, 0x21, 0, 0}, {0x3036, 0x54, 0, 0}, {0x3c07, 0x08, 0, 0},
+ {0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
+ {0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
+ {0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
+ {0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
+ {0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
+ {0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
+ {0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
+ {0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
+ {0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
+ {0x3618, 0x04, 0, 0}, {0x3612, 0x29, 0, 0}, {0x3708, 0x21, 0, 0},
+ {0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
+ {0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
+ {0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
+ {0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
+ {0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x03, 0, 0},
+ {0x4407, 0x04, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
+ {0x3824, 0x02, 0, 0}, {0x5001, 0x83, 0, 70},
+};
+
+/* power-on sensor init reg table */
+static const struct ov5640_mode_info ov5640_mode_init_data = {
+ 0, SUBSAMPLING, 640, 480, ov5640_init_setting_30fps_VGA,
+ ARRAY_SIZE(ov5640_init_setting_30fps_VGA),
+};
+
+static const struct ov5640_mode_info
+ov5640_mode_data[OV5640_NUM_FRAMERATES][OV5640_NUM_MODES] = {
+ {
+ {OV5640_MODE_QCIF_176_144, SUBSAMPLING, 176, 144,
+ ov5640_setting_15fps_QCIF_176_144,
+ ARRAY_SIZE(ov5640_setting_15fps_QCIF_176_144)},
+ {OV5640_MODE_QVGA_320_240, SUBSAMPLING, 320, 240,
+ ov5640_setting_15fps_QVGA_320_240,
+ ARRAY_SIZE(ov5640_setting_15fps_QVGA_320_240)},
+ {OV5640_MODE_VGA_640_480, SUBSAMPLING, 640, 480,
+ ov5640_setting_15fps_VGA_640_480,
+ ARRAY_SIZE(ov5640_setting_15fps_VGA_640_480)},
+ {OV5640_MODE_NTSC_720_480, SUBSAMPLING, 720, 480,
+ ov5640_setting_15fps_NTSC_720_480,
+ ARRAY_SIZE(ov5640_setting_15fps_NTSC_720_480)},
+ {OV5640_MODE_PAL_720_576, SUBSAMPLING, 720, 576,
+ ov5640_setting_15fps_PAL_720_576,
+ ARRAY_SIZE(ov5640_setting_15fps_PAL_720_576)},
+ {OV5640_MODE_XGA_1024_768, SUBSAMPLING, 1024, 768,
+ ov5640_setting_15fps_XGA_1024_768,
+ ARRAY_SIZE(ov5640_setting_15fps_XGA_1024_768)},
+ {OV5640_MODE_720P_1280_720, SUBSAMPLING, 1280, 720,
+ ov5640_setting_15fps_720P_1280_720,
+ ARRAY_SIZE(ov5640_setting_15fps_720P_1280_720)},
+ {OV5640_MODE_1080P_1920_1080, SCALING, 1920, 1080,
+ ov5640_setting_15fps_1080P_1920_1080,
+ ARRAY_SIZE(ov5640_setting_15fps_1080P_1920_1080)},
+ {OV5640_MODE_QSXGA_2592_1944, SCALING, 2592, 1944,
+ ov5640_setting_15fps_QSXGA_2592_1944,
+ ARRAY_SIZE(ov5640_setting_15fps_QSXGA_2592_1944)},
+ }, {
+ {OV5640_MODE_QCIF_176_144, SUBSAMPLING, 176, 144,
+ ov5640_setting_30fps_QCIF_176_144,
+ ARRAY_SIZE(ov5640_setting_30fps_QCIF_176_144)},
+ {OV5640_MODE_QVGA_320_240, SUBSAMPLING, 320, 240,
+ ov5640_setting_30fps_QVGA_320_240,
+ ARRAY_SIZE(ov5640_setting_30fps_QVGA_320_240)},
+ {OV5640_MODE_VGA_640_480, SUBSAMPLING, 640, 480,
+ ov5640_setting_30fps_VGA_640_480,
+ ARRAY_SIZE(ov5640_setting_30fps_VGA_640_480)},
+ {OV5640_MODE_NTSC_720_480, SUBSAMPLING, 720, 480,
+ ov5640_setting_30fps_NTSC_720_480,
+ ARRAY_SIZE(ov5640_setting_30fps_NTSC_720_480)},
+ {OV5640_MODE_PAL_720_576, SUBSAMPLING, 720, 576,
+ ov5640_setting_30fps_PAL_720_576,
+ ARRAY_SIZE(ov5640_setting_30fps_PAL_720_576)},
+ {OV5640_MODE_XGA_1024_768, SUBSAMPLING, 1024, 768,
+ ov5640_setting_30fps_XGA_1024_768,
+ ARRAY_SIZE(ov5640_setting_30fps_XGA_1024_768)},
+ {OV5640_MODE_720P_1280_720, SUBSAMPLING, 1280, 720,
+ ov5640_setting_30fps_720P_1280_720,
+ ARRAY_SIZE(ov5640_setting_30fps_720P_1280_720)},
+ {OV5640_MODE_1080P_1920_1080, SCALING, 1920, 1080,
+ ov5640_setting_30fps_1080P_1920_1080,
+ ARRAY_SIZE(ov5640_setting_30fps_1080P_1920_1080)},
+ {OV5640_MODE_QSXGA_2592_1944, -1, 0, 0, NULL, 0},
+ },
+};
+
+static int ov5640_init_slave_id(struct ov5640_dev *sensor)
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg;
+ u8 buf[3];
+ int ret;
+
+ if (client->addr == OV5640_DEFAULT_SLAVE_ID)
+ return 0;
+
+ buf[0] = OV5640_REG_SLAVE_ID >> 8;
+ buf[1] = OV5640_REG_SLAVE_ID & 0xff;
+ buf[2] = client->addr << 1;
+
+ msg.addr = OV5640_DEFAULT_SLAVE_ID;
+ msg.flags = 0;
+ msg.buf = buf;
+ msg.len = sizeof(buf);
+
+ ret = i2c_transfer(client->adapter, &msg, 1);
+ if (ret < 0) {
+ dev_err(&client->dev, "%s: failed with %d\n", __func__, ret);
+ return ret;
+ }
+
+ }
+
+ return 0;
+}
+
+static int ov5640_write_reg(struct ov5640_dev *sensor, u16 reg, u8 val)
+}
+
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg;
+ u8 buf[3];
+ int ret;
+
+ buf[0] = reg >> 8;
+ buf[1] = reg & 0xff;
+ buf[2] = val;
+
+ msg.addr = client->addr;
+ msg.flags = client->flags;
+ msg.buf = buf;
+ msg.len = sizeof(buf);
+
+ ret = i2c_transfer(client->adapter, &msg, 1);
+ if (ret < 0) {
+ v4l2_err(&sensor->sd, "%s: error: reg=%x, val=%x\n",
+ __func__, reg, val);
+ return ret;
+ }
+
+ }
+
+ return 0;
+}
+
+static int ov5640_read_reg(struct ov5640_dev *sensor, u16 reg, u8 *val)
+}
+
+{
+ struct i2c_client *client = sensor->i2c_client;
+ struct i2c_msg msg[2];
+ u8 buf[2];
+ int ret;
+
+ buf[0] = reg >> 8;
+ buf[1] = reg & 0xff;
+
+ msg[0].addr = client->addr;
+ msg[0].flags = client->flags;
+ msg[0].buf = buf;
+ msg[0].len = sizeof(buf);
+
+ msg[1].addr = client->addr;
+ msg[1].flags = client->flags | I2C_M_RD;
+ msg[1].buf = buf;
+ msg[1].len = 1;
+
+ ret = i2c_transfer(client->adapter, msg, 2);
+ if (ret < 0)
+ return ret;
+
+ *val = buf[0];
+ return 0;
+}
+
+static int ov5640_read_reg16(struct ov5640_dev *sensor, u16 reg, u16 *val)
+}
+
+{
+ u8 hi, lo;
+
+ ov5640_read_reg(sensor, reg, &hi);
+ ov5640_read_reg(sensor, reg+1, &lo);
+
+ *val = ((u16)hi << 8) | (u16)lo;
+ return 0;
+}
+
+static int ov5640_write_reg16(struct ov5640_dev *sensor, u16 reg, u16 val)
+}
+
+{
+ ov5640_write_reg(sensor, reg, val >> 8);
+ ov5640_write_reg(sensor, reg + 1, val & 0xff);
+ return 0;
+}
+
+static int ov5640_mod_reg(struct ov5640_dev *sensor, u16 reg,
+}
+
+ u8 mask, u8 val)
+{
+ u8 readval;
+
+ ov5640_read_reg(sensor, reg, &readval);
+
+ readval &= ~mask;
+ val &= mask;
+ val |= readval;
+
+ ov5640_write_reg(sensor, reg, val);
+ return 0;
+}
+
+/* download ov5640 settings to sensor through i2c */
+}
+
+static int ov5640_load_regs(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *mode)
+{
+ const struct reg_value *regs = mode->reg_data;
+ unsigned int i;
+ u32 delay_ms;
+ u16 reg_addr;
+ u8 mask, val;
+
+ for (i = 0; i < mode->reg_data_size; ++i, ++regs) {
+ delay_ms = regs->delay_ms;
+ reg_addr = regs->reg_addr;
+ val = regs->val;
+ mask = regs->mask;
+
+ if (mask)
+ ov5640_mod_reg(sensor, reg_addr, mask, val);
+ else
+ ov5640_write_reg(sensor, reg_addr, val);
+ if (delay_ms)
+ usleep_range(1000*delay_ms, 1000*delay_ms+100);
+ }
+
+ return 0;
+}
+
+static int ov5640_set_hue(struct ov5640_dev *sensor, int value)
+}
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(0), BIT(0));
+ ov5640_write_reg16(sensor, OV5640_REG_SDE_CTRL1, value);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(0), 0);
+ }
+
+ return 0;
+}
+
+static int ov5640_set_contrast(struct ov5640_dev *sensor, int value)
+}
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(2), BIT(2));
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL5, value & 0xff);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(2), 0);
+ }
+
+ return 0;
+}
+
+static int ov5640_set_saturation(struct ov5640_dev *sensor, int value)
+}
+
+{
+ if (value) {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(1), BIT(1));
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL3, value & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_SDE_CTRL4, value & 0xff);
+ } else {
+ ov5640_mod_reg(sensor, OV5640_REG_SDE_CTRL0, BIT(1), 0);
+ }
+
+ return 0;
+}
+
+static int ov5640_set_white_balance(struct ov5640_dev *sensor, int awb)
+}
+
+{
+ ov5640_mod_reg(sensor, OV5640_REG_AWB_MANUAL_CTRL,
+ BIT(0), awb ? 0 : 1);
+
+ if (!awb) {
+ u16 red = (u16)sensor->ctrls.red_balance->val;
+ u16 blue = (u16)sensor->ctrls.blue_balance->val;
+
+ ov5640_write_reg16(sensor, OV5640_REG_AWB_R_GAIN, red);
+ ov5640_write_reg16(sensor, OV5640_REG_AWB_B_GAIN, blue);
+ }
+
+ return 0;
+}
+
+static int ov5640_set_exposure(struct ov5640_dev *sensor, int exp)
+}
+
+{
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+ bool auto_exposure = (exp == V4L2_EXPOSURE_AUTO);
+
+ if (ctrls->auto_exp->is_new) {
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ BIT(0), auto_exposure ? 0 : BIT(0));
+ }
+
+ if (!auto_exposure && ctrls->exposure->is_new) {
+ u16 max_exp;
+
+ ov5640_read_reg16(sensor, OV5640_REG_AEC_PK_VTS, &max_exp);
+ if (ctrls->exposure->val < max_exp) {
+ u32 exposure = ctrls->exposure->val << 4;
+
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_LO,
+ exposure & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_MED,
+ (exposure >> 8) & 0xff);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_HI,
+ (exposure >> 16) & 0x0f);
+ }
+ }
+
+ return 0;
+}
+
+/* read exposure, in number of line periods */
+}
+
+static int ov5640_get_exposure(struct ov5640_dev *sensor)
+{
+ u8 temp;
+ int exp;
+
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_HI, &temp);
+ exp = ((int)temp & 0x0f) << 16;
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_MED, &temp);
+ exp |= ((int)temp << 8);
+ ov5640_read_reg(sensor, OV5640_REG_AEC_PK_EXPOSURE_LO, &temp);
+ exp |= (int)temp;
+
+ return exp >> 4;
+}
+
+static int ov5640_set_gain(struct ov5640_dev *sensor, int auto_gain)
+{
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+
+ if (ctrls->auto_gain->is_new) {
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ BIT(1), ctrls->auto_gain->val ? 0 : BIT(1));
+ }
+
+ if (!auto_gain && ctrls->gain->is_new) {
+ u16 gain = (u16)ctrls->gain->val;
+
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_PK_REAL_GAIN,
+ gain & 0x3ff);
+ }
+
+ return 0;
+}
+
+static int ov5640_get_gain(struct ov5640_dev *sensor)
+}
+
+{
+ u16 gain;
+
+ ov5640_read_reg16(sensor, OV5640_REG_AEC_PK_REAL_GAIN, &gain);
+
+ return gain & 0x3ff;
+}
+
+static int ov5640_set_agc_aec(struct ov5640_dev *sensor, bool on)
+{
+ ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
+ 0x3, on ? 0 : 0x3);
+ return 0;
+}
+
+static int ov5640_set_test_pattern(struct ov5640_dev *sensor, int value)
+}
+
+{
+ ov5640_mod_reg(sensor, OV5640_REG_PRE_ISP_TEST_SET1,
+ 0xa4, value ? 0xa4 : 0);
+ return 0;
+}
+
+static int ov5640_set_stream(struct ov5640_dev *sensor, bool on)
+}
+
+{
+ ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
+ on ? 0 : BIT(5));
+ ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
+ on ? 0x00 : 0x70);
+ ov5640_write_reg(sensor, OV5640_REG_FRAME_CTRL01,
+ on ? 0x00 : 0x0f);
+ return 0;
+}
+
+static int ov5640_get_sysclk(struct ov5640_dev *sensor)
+}
+
+{
+ /* calculate sysclk */
+ u32 xvclk = sensor->xclk_freq / 10000;
+ u32 multiplier, prediv, VCO, sysdiv, pll_rdiv;
+ u32 sclk_rdiv_map[] = {1, 2, 4, 8};
+ u32 bit_div2x = 1, sclk_rdiv, sysclk;
+ u8 temp1, temp2;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL0, &temp1);
+ temp2 = temp1 & 0x0f;
+ if (temp2 == 8 || temp2 == 10)
+ bit_div2x = temp2 / 2;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL1, &temp1);
+ sysdiv = temp1 >> 4;
+ if (sysdiv == 0)
+ sysdiv = 16;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL2, &temp1);
+ multiplier = temp1;
+
+ ov5640_read_reg(sensor, OV5640_REG_SC_PLL_CTRL3, &temp1);
+ prediv = temp1 & 0x0f;
+ pll_rdiv = ((temp1 >> 4) & 0x01) + 1;
+
+ ov5640_read_reg(sensor, OV5640_REG_SYS_ROOT_DIVIDER, &temp1);
+ temp2 = temp1 & 0x03;
+ sclk_rdiv = sclk_rdiv_map[temp2];
+
+ VCO = xvclk * multiplier / prediv;
+
+ sysclk = VCO / sysdiv / pll_rdiv * 2 / bit_div2x / sclk_rdiv;
+
+ return sysclk;
+}
+
+static int ov5640_set_night_mode(struct ov5640_dev *sensor)
+{
+ /* read HTS from register settings */
+ u8 mode;
+
+ ov5640_read_reg(sensor, OV5640_REG_AEC_CTRL00, &mode);
+ mode &= 0xfb;
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL00, mode);
+ return 0;
+}
+
+static int ov5640_get_hts(struct ov5640_dev *sensor)
+}
+
+{
+ /* read HTS from register settings */
+ u16 hts;
+
+ ov5640_read_reg16(sensor, OV5640_REG_TIMING_HTS, &hts);
+ return hts;
+}
+
+static int ov5640_get_vts(struct ov5640_dev *sensor)
+{
+ u16 vts;
+
+ ov5640_read_reg16(sensor, OV5640_REG_TIMING_VTS, &vts);
+ return vts;
+}
+
+static int ov5640_set_vts(struct ov5640_dev *sensor, int vts)
+{
+ ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, vts);
+ return 0;
+}
+
+static int ov5640_get_light_freq(struct ov5640_dev *sensor)
+}
+
+{
+ /* get banding filter value */
+ u8 temp, temp1;
+ int light_freq = 0;
+
+ ov5640_read_reg(sensor, OV5640_REG_HZ5060_CTRL01, &temp);
+
+ if (temp & 0x80) {
+ /* manual */
+ ov5640_read_reg(sensor, OV5640_REG_HZ5060_CTRL00, &temp1);
+ if (temp1 & 0x04) {
+ /* 50Hz */
+ light_freq = 50;
+ } else {
+ /* 60Hz */
+ light_freq = 60;
+ }
+ } else {
+ /* auto */
+ ov5640_read_reg(sensor, OV5640_REG_SIGMADELTA_CTRL0C,
+ &temp1);
+ if (temp1 & 0x01) {
+ /* 50Hz */
+ light_freq = 50;
+ } else {
+ /* 60Hz */
+ }
+ }
+
+ return light_freq;
+}
+
+static int ov5640_set_bandingfilter(struct ov5640_dev *sensor)
+{
+ u32 band_step60, max_band60, band_step50, max_band50, prev_vts;
+ int ret;
+
+ /* read preview PCLK */
+ ret = ov5640_get_sysclk(sensor);
+ if (ret < 0)
+ return ret;
+ sensor->prev_sysclk = ret;
+ return ret;
+ /* read preview HTS */
+ ret = ov5640_get_hts(sensor);
+ if (ret < 0)
+ return ret;
+ sensor->prev_hts = ret;
+ return ret;
+
+ /* read preview VTS */
+ ret = ov5640_get_vts(sensor);
+ if (ret < 0)
+ return ret;
+ prev_vts = ret;
+ return ret;
+
+ /* calculate banding filter */
+ /* 60Hz */
+ band_step60 = sensor->prev_sysclk * 100 / sensor->prev_hts * 100/120;
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_B60_STEP, band_step60);
+
+ max_band60 = (int)((prev_vts-4)/band_step60);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0D, max_band60);
+
+ /* 50Hz */
+ band_step50 = sensor->prev_sysclk * 100 / sensor->prev_hts;
+ ov5640_write_reg16(sensor, OV5640_REG_AEC_B50_STEP, band_step50);
+
+ max_band50 = (int)((prev_vts-4)/band_step50);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0E, max_band50);
+
+ return 0;
+}
+
+static int ov5640_set_ae_target(struct ov5640_dev *sensor, int target)
+}
+
+{
+ /* stable in high */
+ u32 fast_high, fast_low;
+
+ sensor->ae_low = target * 23 / 25; /* 0.92 */
+ sensor->ae_high = target * 27 / 25; /* 1.08 */
+
+ fast_high = sensor->ae_high << 1;
+ if (fast_high > 255)
+ fast_high = 255;
+
+ fast_low = sensor->ae_low >> 1;
+
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL0F, sensor->ae_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL10, sensor->ae_low);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1B, sensor->ae_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1E, sensor->ae_low);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL11, fast_high);
+ ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1F, fast_low);
+
+ return 0;
+}
+
+static int ov5640_binning_on(struct ov5640_dev *sensor)
+}
+
+{
+ u8 temp;
+
+ ov5640_read_reg(sensor, OV5640_REG_TIMING_TC_REG21, &temp);
+ temp &= 0xfe;
+
+ return temp ? 1 : 0;
+}
+
+static int ov5640_set_virtual_channel(struct ov5640_dev *sensor)
+{
+ u8 temp, channel = OV5640_MIPI_VC;
+
+ ov5640_read_reg(sensor, OV5640_REG_DEBUG_MODE, &temp);
+ temp &= ~(3 << 6);
+ temp |= (channel << 6);
+ ov5640_write_reg(sensor, OV5640_REG_DEBUG_MODE, temp);
+
+ return 0;
+}
+
+static const struct ov5640_mode_info *
+}
+
+ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
+ int width, int height, bool nearest)
+{
+ const struct ov5640_mode_info *mode = NULL;
+ int i;
+
+ for (i = OV5640_NUM_MODES - 1; i >= 0; i--) {
+ mode = &ov5640_mode_data[fr][i];
+
+ if (!mode->reg_data)
+ continue;
+
+ if ((nearest && mode->width <= width &&
+ mode->height <= height) ||
+ (!nearest && mode->width == width &&
+ mode->height == height))
+ break;
+ }
+
+ if (nearest && i < 0)
+ mode = &ov5640_mode_data[fr][0];
+
+ return mode;
+}
+
+/*
+ * sensor changes between scaling and subsampling, go through
+ * exposure calculation
+ */
+static int ov5640_set_mode_exposure_calc(
+ struct ov5640_dev *sensor, const struct ov5640_mode_info *mode)
+{
+ u32 prev_shutter, prev_gain16;
+ u32 cap_shutter, cap_gain16;
+ u32 cap_sysclk, cap_hts, cap_vts;
+ u32 light_freq, cap_bandfilt, cap_maxband;
+ u32 cap_gain16_shutter;
+ u8 average;
+ int ret;
+
+ if (mode->reg_data == NULL)
+ return -EINVAL;
+
+ /* turn off AE/AG */
+ ret = ov5640_set_agc_aec(sensor, false);
+ if (ret < 0)
+ return ret;
+
+ /* read preview shutter */
+ ret = ov5640_get_exposure(sensor);
+ if (ret < 0)
+ return ret;
+ prev_shutter = ret;
+ return ret;
+ ret = ov5640_binning_on(sensor);
+ if (ret < 0)
+ return ret;
+ if (ret && mode->id != OV5640_MODE_720P_1280_720 &&
+ return ret;
+ mode->id != OV5640_MODE_1080P_1920_1080)
+ prev_shutter *= 2;
+
+ /* read preview gain */
+ ret = ov5640_get_gain(sensor);
+ if (ret < 0)
+ return ret;
+ prev_gain16 = ret;
+ return ret;
+
+ /* get average */
+ ov5640_read_reg(sensor, OV5640_REG_AVG_READOUT, &average);
+
+ /* turn off night mode for capture */
+ ret = ov5640_set_night_mode(sensor);
+ if (ret < 0)
+ return ret;
+
+ /* Write capture setting */
+ ret = ov5640_load_regs(sensor, mode);
+ if (ret < 0)
+ return ret;
+
+ /* read capture VTS */
+ ret = ov5640_get_vts(sensor);
+ if (ret < 0)
+ return ret;
+ cap_vts = ret;
+ return ret;
+ ret = ov5640_get_hts(sensor);
+ if (ret < 0)
+ return ret;
+ cap_hts = ret;
+ return ret;
+ ret = ov5640_get_sysclk(sensor);
+ if (ret < 0)
+ return ret;
+ cap_sysclk = ret;
+ return ret;
+
+ /* calculate capture banding filter */
+ ret = ov5640_get_light_freq(sensor);
+ if (ret < 0)
+ return ret;
+ light_freq = ret;
+ return ret;
+
+ if (light_freq == 60) {
+ /* 60Hz */
+ cap_bandfilt = cap_sysclk * 100 / cap_hts * 100 / 120;
+ } else {
+ /* 50Hz */
+ cap_bandfilt = cap_sysclk * 100 / cap_hts;
+ }
+ cap_maxband = (int)((cap_vts - 4) / cap_bandfilt);
+
+ /* calculate capture shutter/gain16 */
+ if (average > sensor->ae_low && average < sensor->ae_high) {
+ /* in stable range */
+ cap_gain16_shutter =
+ prev_gain16 * prev_shutter *
+ cap_sysclk / sensor->prev_sysclk *
+ sensor->prev_hts / cap_hts *
+ sensor->ae_target / average;
+ } else {
+ cap_gain16_shutter =
+ prev_gain16 * prev_shutter *
+ cap_sysclk / sensor->prev_sysclk *
+ sensor->prev_hts / cap_hts;
+ }
+
+ /* gain to shutter */
+ if (cap_gain16_shutter < (cap_bandfilt * 16)) {
+ /* shutter < 1/100 */
+ cap_shutter = cap_gain16_shutter / 16;
+ if (cap_shutter < 1)
+ cap_shutter = 1;
+
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ if (cap_gain16 < 16)
+ cap_gain16 = 16;
+ } else {
+ if (cap_gain16_shutter > (cap_bandfilt * cap_maxband * 16)) {
+ /* exposure reach max */
+ cap_shutter = cap_bandfilt * cap_maxband;
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ } else {
+ /* 1/100 < (cap_shutter = n/100) =< max */
+ cap_shutter =
+ ((int)(cap_gain16_shutter / 16 / cap_bandfilt))
+ * cap_bandfilt;
+ cap_gain16 = cap_gain16_shutter / cap_shutter;
+ }
+ }
+
+ /* write capture gain */
+ ret = ov5640_set_gain(sensor, cap_gain16);
+ if (ret < 0)
+ return ret;
+
+ /* write capture shutter */
+ if (cap_shutter > (cap_vts - 4)) {
+ cap_vts = cap_shutter + 4;
+ ret = ov5640_set_vts(sensor, cap_vts);
+ if (ret < 0)
+ return ret;
+ }
+
+ return ov5640_set_exposure(sensor, cap_shutter);
+ }
+
+}
+
+/*
+ * if sensor changes inside scaling or subsampling
+ * change mode directly
+ */
+static int ov5640_set_mode_direct(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *mode)
+{
+ int ret;
+
+ if (mode->reg_data == NULL)
+ return -EINVAL;
+
+ /* turn off AE/AG */
+ ret = ov5640_set_agc_aec(sensor, false);
+ if (ret < 0)
+ return ret;
+
+ /* Write capture setting */
+ ret = ov5640_load_regs(sensor, mode);
+ if (ret < 0)
+ return ret;
+
+ return ov5640_set_agc_aec(sensor, true);
+}
+
+static int ov5640_set_mode(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *orig_mode)
+{
+ const struct ov5640_mode_info *mode = sensor->current_mode;
+ enum ov5640_downsize_mode dn_mode, orig_dn_mode;
+ int ret;
+
+ dn_mode = mode->dn_mode;
+ orig_dn_mode = orig_mode->dn_mode;
+
+ if ((dn_mode == SUBSAMPLING && orig_dn_mode == SCALING) ||
+ (dn_mode == SCALING && orig_dn_mode == SUBSAMPLING)) {
+ /*
+ * change between subsampling and scaling
+ * go through exposure calucation
+ */
+ ret = ov5640_set_mode_exposure_calc(sensor, mode);
+ } else {
+ /*
+ * change inside subsampling or scaling
+ * download firmware directly
+ */
+ ret = ov5640_set_mode_direct(sensor, mode);
+ }
+
+ if (ret < 0)
+ return ret;
+
+ ret = ov5640_set_ae_target(sensor, sensor->ae_target);
+ if (ret < 0)
+ return ret;
+ ret = ov5640_get_light_freq(sensor);
+ return ret;
+ if (ret < 0)
+ return ret;
+ ret = ov5640_set_bandingfilter(sensor);
+ return ret;
+ if (ret < 0)
+ return ret;
+ ret = ov5640_set_virtual_channel(sensor);
+ return ret;
+ if (ret < 0)
+ return ret;
+
+ sensor->pending_mode_change = false;
+
+ return 0;
+}
+
+/* restore the last set video mode after chip power-on */
+}
+
+static int ov5640_restore_mode(struct ov5640_dev *sensor)
+{
+ int ret;
+
+ /* first load the initial register values */
+ ret = ov5640_load_regs(sensor, &ov5640_mode_init_data);
+ if (ret < 0)
+ return ret;
+
+ /* now restore the last capture mode */
+ return ov5640_set_mode(sensor, &ov5640_mode_init_data);
+}
+
+static void ov5640_power(struct ov5640_dev *sensor, bool enable)
+{
+ if (sensor->pwdn_gpio)
+ gpiod_set_value(sensor->pwdn_gpio, enable ? 0 : 1);
+}
+
+static void ov5640_reset(struct ov5640_dev *sensor)
+{
+ if (!sensor->reset_gpio)
+ return;
+
+ gpiod_set_value(sensor->reset_gpio, 0);
+
+ /* camera power cycle */
+ ov5640_power(sensor, false);
+ usleep_range(5000, 10000);
+ ov5640_power(sensor, true);
+ usleep_range(5000, 10000);
+
+ gpiod_set_value(sensor->reset_gpio, 1);
+ usleep_range(1000, 2000);
+
+ gpiod_set_value(sensor->reset_gpio, 0);
+ usleep_range(5000, 10000);
+}
+
+static int ov5640_set_power(struct ov5640_dev *sensor, bool on)
+{
+ int ret = 0;
+
+ if (on) {
+
+ clk_prepare_enable(sensor->xclk);
+
+ ret = regulator_bulk_enable(OV5640_NUM_SUPPLIES,
+ sensor->supplies);
+ if (ret)
+ goto xclk_off;
+
+ ov5640_reset(sensor);
+ ov5640_power(sensor, true);
+
+ ret = ov5640_init_slave_id(sensor);
+ if (ret)
+ goto power_off;
+
+ ret = ov5640_restore_mode(sensor);
+ if (ret)
+ goto power_off;
+
+ /*
+ * start streaming briefly followed by stream off in
+ * order to coax the clock lane into LP-11 state.
+ */
+ ov5640_set_stream(sensor, true);
+ usleep_range(1000, 2000);
+ ov5640_set_stream(sensor, false);
+
+ return 0;
+ }
+
+power_off:
+ }
+
+ ov5640_power(sensor, false);
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+xclk_off:
+ clk_disable_unprepare(sensor->xclk);
+ return ret;
+}
+
+/* --------------- Subdev Operations --------------- */
+}
+
+
+static int ov5640_s_power(struct v4l2_subdev *sd, int on)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ int ret = 0;
+
+ mutex_lock(&sensor->lock);
+
+
+ /*
+ * If the power count is modified from 0 to != 0 or from != 0 to 0,
+ * update the power state.
+ */
+ if (sensor->power_count == !on) {
+ /*
+ * If the power count is modified from 0 to != 0 or from != 0 to 0,
+ * update the power state.
+ */
+ ret = ov5640_set_power(sensor, !!on);
+ if (ret)
+ goto out;
+ }
+
+ }
+
+ /* Update the power count. */
+ sensor->power_count += on ? 1 : -1;
+ WARN_ON(sensor->power_count < 0);
+out:
+ mutex_unlock(&sensor->lock);
+
+ if (on && !ret && sensor->power_count == 1) {
+ /* restore controls */
+ ret = v4l2_ctrl_handler_setup(&sensor->ctrls.handler);
+ }
+
+ return ret;
+}
+
+static int ov5640_try_frame_interval(struct ov5640_dev *sensor,
+ return ret;
+}
+
+ struct v4l2_fract *fi,
+ u32 width, u32 height)
+{
+ const struct ov5640_mode_info *mode;
+ u32 minfps, maxfps, fps;
+ int ret;
+
+ minfps = ov5640_framerates[OV5640_15_FPS];
+ maxfps = ov5640_framerates[OV5640_30_FPS];
+
+ if (fi->numerator == 0) {
+ fi->denominator = maxfps;
+ fi->numerator = 1;
+ return OV5640_30_FPS;
+ }
+
+ fps = DIV_ROUND_CLOSEST(fi->denominator, fi->numerator);
+
+ fi->numerator = 1;
+ if (fps > maxfps)
+ fi->denominator = maxfps;
+ else if (fps < minfps)
+ fi->denominator = minfps;
+ else if (2 * fps >= 2 * minfps + (maxfps - minfps))
+ fi->denominator = maxfps;
+ else
+ fi->denominator = minfps;
+
+ ret = (fi->denominator == minfps) ? OV5640_15_FPS : OV5640_30_FPS;
+
+ mode = ov5640_find_mode(sensor, ret, width, height, false);
+ return mode ? ret : -EINVAL;
+}
+
+static int ov5640_g_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_captureparm *cparm = &a->parm.capture;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+
+ mutex_lock(&sensor->lock);
+
+ /* This is the only case currently handled. */
+ memset(a, 0, sizeof(*a));
+ a->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ cparm->capability = sensor->streamcap.capability;
+ cparm->timeperframe = sensor->streamcap.timeperframe;
+ cparm->capturemode = sensor->streamcap.capturemode;
+
+ mutex_unlock(&sensor->lock);
+
+ return 0;
+}
+
+static int ov5640_s_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
+}
+
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_fract *tpf = &a->parm.capture.timeperframe;
+ const struct ov5640_mode_info *mode;
+ int frame_rate, ret = 0;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ mode = sensor->current_mode;
+ }
+
+
+ frame_rate = ov5640_try_frame_interval(sensor, tpf,
+ mode->width, mode->height);
+ if (frame_rate < 0) {
+ ret = frame_rate;
+ goto out;
+ }
+
+ sensor->current_fr = frame_rate;
+ }
+
+ sensor->streamcap.timeperframe = *tpf;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+ return 0;
+}
+
+static int ov5640_get_fmt(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_mbus_framefmt *fmt;
+
+ if (format->pad != 0)
+
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY)
+ fmt = v4l2_subdev_get_try_format(&sensor->sd, cfg,
+ format->pad);
+ else
+ fmt = &sensor->fmt;
+
+ format->format = *fmt;
+
+ mutex_unlock(&sensor->lock);
+
+ return 0;
+}
+
+static int ov5640_try_fmt_internal(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_mbus_framefmt *fmt,
+ enum ov5640_frame_rate fr,
+ const struct ov5640_mode_info **new_mode)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ const struct ov5640_mode_info *mode;
+
+ mode = ov5640_find_mode(sensor, fr, fmt->width, fmt->height, true);
+ if (!mode)
+ return -EINVAL;
+
+ fmt->width = mode->width;
+ fmt->height = mode->height;
+ fmt->code = sensor->fmt.code;
+
+ if (new_mode)
+ *new_mode = mode;
+ return 0;
+}
+
+static int ov5640_set_fmt(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *format)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ const struct ov5640_mode_info *new_mode;
+ int ret;
+
+ if (format->pad != 0)
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ ret = ov5640_try_fmt_internal(sd, &format->format,
+ }
+
+ sensor->current_fr, &new_mode);
+ if (ret)
+ goto out;
+
+ if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
+ struct v4l2_mbus_framefmt *fmt =
+ v4l2_subdev_get_try_format(sd, cfg, 0);
+
+ *fmt = format->format;
+ goto out;
+ }
+
+ sensor->current_mode = new_mode;
+ }
+
+ sensor->fmt = format->format;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+ return ret;
+}
+
+
+}
+
+/*
+ * Sensor Controls.
+ */
+
+static int ov5640_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ mutex_lock(&sensor->lock);
+
+ switch (ctrl->id) {
+ case V4L2_CID_AUTOGAIN:
+ switch (ctrl->id) {
+ if (!ctrl->val)
+ goto out;
+ sensor->ctrls.gain->val = ov5640_get_gain(sensor);
+ break;
+ case V4L2_CID_EXPOSURE_AUTO:
+ if (ctrl->val == V4L2_EXPOSURE_MANUAL)
+ goto out;
+ sensor->ctrls.exposure->val = ov5640_get_exposure(sensor);
+ break;
+ }
+
+out:
+ mutex_unlock(&sensor->lock);
+ return 0;
+}
+
+static int ov5640_s_ctrl(struct v4l2_ctrl *ctrl)
+}
+
+{
+ struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ int ret = 0;
+
+ mutex_lock(&sensor->lock);
+
+
+ /*
+ * If the device is not powered up by the host driver do
+ * not apply any controls to H/W at this time. Instead
+ * the controls will be restored right after power-up.
+ */
+ if (sensor->power_count == 0)
+ goto out;
+
+ switch (ctrl->id) {
+ case V4L2_CID_AUTOGAIN:
+ ret = ov5640_set_gain(sensor, ctrl->val);
+ break;
+ case V4L2_CID_EXPOSURE_AUTO:
+ ret = ov5640_set_exposure(sensor, ctrl->val);
+ break;
+ case V4L2_CID_AUTO_WHITE_BALANCE:
+ ret = ov5640_set_white_balance(sensor, ctrl->val);
+ break;
+ case V4L2_CID_HUE:
+ ret = ov5640_set_hue(sensor, ctrl->val);
+ break;
+ case V4L2_CID_CONTRAST:
+ ret = ov5640_set_contrast(sensor, ctrl->val);
+ break;
+ case V4L2_CID_SATURATION:
+ ret = ov5640_set_saturation(sensor, ctrl->val);
+ break;
+ case V4L2_CID_TEST_PATTERN:
+ ret = ov5640_set_test_pattern(sensor, ctrl->val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+out:
+ mutex_unlock(&sensor->lock);
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops ov5640_ctrl_ops = {
+}
+
+ .g_volatile_ctrl = ov5640_g_volatile_ctrl,
+ .s_ctrl = ov5640_s_ctrl,
+};
+
+static const char * const test_pattern_menu[] = {
+ "Disabled",
+ "Color bars",
+};
+
+static int ov5640_init_controls(struct ov5640_dev *sensor)
+{
+ const struct v4l2_ctrl_ops *ops = &ov5640_ctrl_ops;
+ struct ov5640_ctrls *ctrls = &sensor->ctrls;
+ struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+ int ret;
+
+ v4l2_ctrl_handler_init(hdl, 32);
+
+ /* Auto/manual white balance */
+ ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops,
+ V4L2_CID_AUTO_WHITE_BALANCE,
+ 0, 1, 1, 1);
+ ctrls->blue_balance = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_BLUE_BALANCE,
+ 0, 4095, 1, 0);
+ ctrls->red_balance = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_RED_BALANCE,
+ 0, 4095, 1, 0);
+ /* Auto/manual exposure */
+ ctrls->auto_exp = v4l2_ctrl_new_std_menu(hdl, ops,
+ V4L2_CID_EXPOSURE_AUTO,
+ V4L2_EXPOSURE_MANUAL, 0,
+ V4L2_EXPOSURE_AUTO);
+ ctrls->exposure = v4l2_ctrl_new_std(hdl, ops,
+ V4L2_CID_EXPOSURE_ABSOLUTE,
+ 0, 65535, 1, 0);
+ /* Auto/manual gain */
+ ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN,
+ 0, 1, 1, 1);
+ ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
+ 0, 1023, 1, 0);
+
+ ctrls->saturation = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_SATURATION,
+ 0, 255, 1, 64);
+ ctrls->hue = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HUE,
+ 0, 359, 1, 0);
+ ctrls->contrast = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_CONTRAST,
+ 0, 255, 1, 0);
+ ctrls->test_pattern =
+ v4l2_ctrl_new_std_menu_items(hdl, ops, V4L2_CID_TEST_PATTERN,
+ ARRAY_SIZE(test_pattern_menu) - 1,
+ 0, 0, test_pattern_menu);
+
+ if (hdl->error) {
+ ret = hdl->error;
+ goto free_ctrls;
+ }
+
+ ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
+ ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ v4l2_ctrl_auto_cluster(3, &ctrls->auto_wb, 0, false);
+ v4l2_ctrl_auto_cluster(2, &ctrls->auto_gain, 0, true);
+ v4l2_ctrl_auto_cluster(2, &ctrls->auto_exp, 1, true);
+
+ sensor->sd.ctrl_handler = hdl;
+ return 0;
+
+free_ctrls:
+ v4l2_ctrl_handler_free(hdl);
+ return ret;
+}
+
+static int ov5640_enum_frame_size(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->pad != 0)
+ return -EINVAL;
+ if (fse->index >= OV5640_NUM_MODES)
+ return -EINVAL;
+
+ fse->min_width = fse->max_width =
+ ov5640_mode_data[0][fse->index].width;
+ fse->min_height = fse->max_height =
+ ov5640_mode_data[0][fse->index].height;
+
+ return 0;
+}
+
+static int ov5640_enum_frame_interval(
+}
+
+ struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_interval_enum *fie)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_fract tpf;
+ int ret;
+
+ if (fie->pad != 0)
+ return -EINVAL;
+ if (fie->index >= OV5640_NUM_FRAMERATES)
+ return -EINVAL;
+
+ tpf.numerator = 1;
+ tpf.denominator = ov5640_framerates[fie->index];
+
+ ret = ov5640_try_frame_interval(sensor, &tpf,
+ fie->width, fie->height);
+ if (ret < 0)
+ return -EINVAL;
+
+ fie->interval = tpf;
+ return 0;
+}
+
+static int ov5640_g_frame_interval(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+{
+
+ mutex_lock(&sensor->lock);
+ fi->interval = sensor->streamcap.timeperframe;
+ mutex_unlock(&sensor->lock);
+
+ return 0;
+}
+
+static int ov5640_s_frame_interval(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_frame_interval *fi)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+{
+ const struct ov5640_mode_info *mode;
+ int frame_rate, ret = 0;
+
+ if (fi->pad != 0)
+ return -EINVAL;
+
+ mutex_lock(&sensor->lock);
+
+ if (sensor->streaming) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ mode = sensor->current_mode;
+ }
+
+
+ frame_rate = ov5640_try_frame_interval(sensor, &fi->interval,
+ mode->width, mode->height);
+ if (frame_rate < 0)
+ frame_rate = OV5640_15_FPS;
+
+ sensor->current_fr = frame_rate;
+ sensor->streamcap.timeperframe = fi->interval;
+ sensor->pending_mode_change = true;
+out:
+ mutex_unlock(&sensor->lock);
+ return ret;
+}
+
+static int ov5640_enum_mbus_code(struct v4l2_subdev *sd,
+}
+
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+
+ if (code->pad != 0)
+ return -EINVAL;
+ if (code->index != 0)
+ return -EINVAL;
+
+ code->code = sensor->fmt.code;
+
+ return 0;
+}
+
+static int ov5640_s_stream(struct v4l2_subdev *sd, int enable)
+}
+
+{
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+ int ret = 0;
+
+ mutex_lock(&sensor->lock);
+
+
+ if (sensor->streaming == !enable) {
+ if (enable && sensor->pending_mode_change) {
+ ret = ov5640_set_mode(sensor, sensor->current_mode);
+ if (ret)
+ goto out;
+ }
+
+ ret = ov5640_set_stream(sensor, enable);
+ }
+
+ if (!ret)
+ sensor->streaming = enable;
+ }
+out:
+ mutex_unlock(&sensor->lock);
+ return ret;
+}
+
+static const struct v4l2_subdev_core_ops ov5640_core_ops = {
+}
+
+ .s_power = ov5640_s_power,
+};
+
+static const struct v4l2_subdev_video_ops ov5640_video_ops = {
+ .s_parm = ov5640_s_parm,
+ .g_parm = ov5640_g_parm,
+ .g_frame_interval = ov5640_g_frame_interval,
+ .s_frame_interval = ov5640_s_frame_interval,
+ .s_stream = ov5640_s_stream,
+};
+
+static const struct v4l2_subdev_pad_ops ov5640_pad_ops = {
+ .enum_mbus_code = ov5640_enum_mbus_code,
+ .get_fmt = ov5640_get_fmt,
+ .set_fmt = ov5640_set_fmt,
+ .enum_frame_size = ov5640_enum_frame_size,
+ .enum_frame_interval = ov5640_enum_frame_interval,
+};
+
+static const struct v4l2_subdev_ops ov5640_subdev_ops = {
+ .core = &ov5640_core_ops,
+ .video = &ov5640_video_ops,
+ .pad = &ov5640_pad_ops,
+};
+
+static int ov5640_get_regulators(struct ov5640_dev *sensor)
+{
+ int i;
+
+ for (i = 0; i < OV5640_NUM_SUPPLIES; i++)
+ sensor->supplies[i].supply = ov5640_supply_name[i];
+
+ return devm_regulator_bulk_get(&sensor->i2c_client->dev,
+ OV5640_NUM_SUPPLIES,
+ sensor->supplies);
+}
+
+static int ov5640_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &client->dev;
+ struct device_node *endpoint;
+ struct ov5640_dev *sensor;
+ int ret;
+
+ sensor = devm_kzalloc(dev, sizeof(*sensor), GFP_KERNEL);
+ if (!sensor)
+ return -ENOMEM;
+
+ sensor->i2c_client = client;
+ sensor->fmt.code = MEDIA_BUS_FMT_UYVY8_2X8;
+ sensor->fmt.width = 640;
+ sensor->fmt.height = 480;
+ sensor->fmt.field = V4L2_FIELD_NONE;
+ sensor->streamcap.capability = V4L2_CAP_TIMEPERFRAME;
+ sensor->streamcap.capturemode = 0;
+ sensor->streamcap.timeperframe.denominator =
+ ov5640_framerates[OV5640_30_FPS];
+ sensor->streamcap.timeperframe.numerator = 1;
+ sensor->current_fr = OV5640_30_FPS;
+ sensor->current_mode =
+ &ov5640_mode_data[OV5640_30_FPS][OV5640_MODE_VGA_640_480];
+ sensor->pending_mode_change = true;
+
+ sensor->ae_target = 52;
+
+ endpoint = of_graph_get_next_endpoint(client->dev.of_node, NULL);
+ if (!endpoint) {
+ dev_err(dev, "endpoint node not found\n");
+ return -EINVAL;
+ }
+
+ v4l2_of_parse_endpoint(endpoint, &sensor->ep);
+ }
+
+ of_node_put(endpoint);
+
+ if (sensor->ep.bus_type != V4L2_MBUS_CSI2) {
+ dev_err(dev, "invalid bus type, must be MIPI CSI2\n");
+ return -EINVAL;
+ }
+
+ /* get system clock (xclk) */
+ }
+
+ sensor->xclk = devm_clk_get(dev, "xclk");
+ if (IS_ERR(sensor->xclk)) {
+ dev_err(dev, "failed to get xclk\n");
+ return PTR_ERR(sensor->xclk);
+ }
+
+ sensor->xclk_freq = clk_get_rate(sensor->xclk);
+ if (sensor->xclk_freq < OV5640_XCLK_MIN ||
+ sensor->xclk_freq > OV5640_XCLK_MAX) {
+ dev_err(dev, "xclk frequency out of range: %d Hz\n",
+ sensor->xclk_freq);
+ return -EINVAL;
+ }
+
+ /* request optional power down pin */
+ }
+
+ sensor->pwdn_gpio = devm_gpiod_get_optional(dev, "powerdown",
+ GPIOD_OUT_HIGH);
+ /* request optional reset pin */
+ sensor->reset_gpio = devm_gpiod_get_optional(dev, "reset",
+ GPIOD_OUT_HIGH);
+
+ v4l2_i2c_subdev_init(&sensor->sd, client, &ov5640_subdev_ops);
+
+ sensor->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
+ sensor->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
+ ret = media_entity_pads_init(&sensor->sd.entity, 1, &sensor->pad);
+ if (ret)
+ return ret;
+
+ ret = ov5640_get_regulators(sensor);
+ return ret;
+
+ if (ret)
+ return ret;
+
+ mutex_init(&sensor->lock);
+ return ret;
+
+
+ ret = ov5640_init_controls(sensor);
+ if (ret)
+ goto entity_cleanup;
+
+ ret = v4l2_async_register_subdev(&sensor->sd);
+ if (ret)
+ goto free_ctrls;
+
+ return 0;
+
+free_ctrls:
+ return 0;
+
+ v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+entity_cleanup:
+ mutex_destroy(&sensor->lock);
+ media_entity_cleanup(&sensor->sd.entity);
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+ return ret;
+}
+
+static int ov5640_remove(struct i2c_client *client)
+}
+
+{
+ struct v4l2_subdev *sd = i2c_get_clientdata(client);
+ struct ov5640_dev *sensor = to_ov5640_dev(sd);
+
+ regulator_bulk_disable(OV5640_NUM_SUPPLIES, sensor->supplies);
+
+ v4l2_async_unregister_subdev(&sensor->sd);
+ mutex_destroy(&sensor->lock);
+ media_entity_cleanup(&sensor->sd.entity);
+ v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+
+ return 0;
+}
+
+static const struct i2c_device_id ov5640_id[] = {
+}
+
+ {"ov5640", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, ov5640_id);
+
+static const struct of_device_id ov5640_dt_ids[] = {
+ { .compatible = "ovti,ov5640" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, ov5640_dt_ids);
+};
+
+static struct i2c_driver ov5640_i2c_driver = {
+ .driver = {
+ .name = "ov5640",
+ .of_match_table = ov5640_dt_ids,
+ },
+ .id_table = ov5640_id,
+ .probe = ov5640_probe,
+ .remove = ov5640_remove,
+};
+
+module_i2c_driver(ov5640_i2c_driver);
+
+MODULE_DESCRIPTION("OV5640 MIPI Camera Subdev Driver");
Steve Longerbeam
Mar 10, 2017, 12:10:05 AM3/10/17
to
Add a NEW_FRAME_BEFORE_EOF event to signal that a video capture or
output device has signaled a new frame is ready before a previous
frame has completed reception or transmission. This usually indicates
a DMA read/write channel is having trouble gaining bus access.
--- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
+++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
@@ -203,6 +203,12 @@ call.
+ - 8
+ completed reception or transmission. This usually indicates a
+ DMA read/write channel is having trouble gaining bus access.
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -460,6 +460,7 @@ replace define V4L2_EVENT_FRAME_SYNC event-type
+replace define V4L2_EVENT_NEW_FRAME_BEFORE_EOF event-type
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2132,6 +2132,7 @@ struct v4l2_streamparm {
+#define V4L2_EVENT_NEW_FRAME_BEFORE_EOF 8
output device has signaled a new frame is ready before a previous
frame has completed reception or transmission. This usually indicates
a DMA read/write channel is having trouble gaining bus access.
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 6 ++++++
Documentation/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 8 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
index dc77363..54bc7ae 100644
Documentation/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 8 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
--- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
+++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
@@ -203,6 +203,12 @@ call.
has measured an interval between the reception or transmit
completion of two consecutive frames of video that is outside
the nominal frame interval by some tolerance value.
+ * - ``V4L2_EVENT_NEW_FRAME_BEFORE_EOF``
completion of two consecutive frames of video that is outside
the nominal frame interval by some tolerance value.
+ - 8
+ - This event is triggered when the video capture or output device
+ has signaled a new frame is ready before a previous frame has
+ completed reception or transmission. This usually indicates a
+ DMA read/write channel is having trouble gaining bus access.
* - ``V4L2_EVENT_PRIVATE_START``
- 0x08000000
- Base event number for driver-private events.
diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
index c7d8fad..be6f332 100644
- 0x08000000
- Base event number for driver-private events.
diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -460,6 +460,7 @@ replace define V4L2_EVENT_FRAME_SYNC event-type
replace define V4L2_EVENT_SOURCE_CHANGE event-type
replace define V4L2_EVENT_MOTION_DET event-type
replace define V4L2_EVENT_FRAME_INTERVAL_ERROR event-type
replace define V4L2_EVENT_MOTION_DET event-type
+replace define V4L2_EVENT_NEW_FRAME_BEFORE_EOF event-type
replace define V4L2_EVENT_PRIVATE_START event-type
replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index cf5a0d0..f54a82a 100644
replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2132,6 +2132,7 @@ struct v4l2_streamparm {
#define V4L2_EVENT_SOURCE_CHANGE 5
#define V4L2_EVENT_MOTION_DET 6
#define V4L2_EVENT_FRAME_INTERVAL_ERROR 7
#define V4L2_EVENT_MOTION_DET 6
+#define V4L2_EVENT_NEW_FRAME_BEFORE_EOF 8
Steve Longerbeam
Mar 10, 2017, 12:10:05 AM3/10/17
to
Add pinctrl groups for both GPT input capture channels.
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 21dea5f..1212f82 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -456,6 +456,18 @@
>;
};
+ pinctrl_gpt_input_capture0: gptinputcapture0grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1 0x1b0b0
+ >;
+ };
+
+ pinctrl_gpt_input_capture1: gptinputcapture1grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2 0x1b0b0
+ >;
+ };
+
pinctrl_spdif: spdifgrp {
fsl,pins = <
MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
--
2.7.4
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 21dea5f..1212f82 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -456,6 +456,18 @@
>;
};
+ pinctrl_gpt_input_capture0: gptinputcapture0grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1 0x1b0b0
+ >;
+ };
+
+ pinctrl_gpt_input_capture1: gptinputcapture1grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2 0x1b0b0
+ >;
+ };
+
pinctrl_spdif: spdifgrp {
fsl,pins = <
MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
--
2.7.4
Hans Verkuil
Mar 10, 2017, 6:50:07 AM3/10/17
to
variant takes.
Regards,
Hans
Hans Verkuil
Mar 10, 2017, 6:50:11 AM3/10/17
to
On 10/03/17 05:53, Steve Longerbeam wrote:
> This adds a header file for use by userspace programs wanting to interact
> with the i.MX media driver. It defines custom v4l2 controls for the
> i.MX v4l2 subdevices.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
I would not export this while the imx driver is in staging.
Also, traditionally exported media headers are in linux, not media.
I would like to have a discussion about that before deciding where to
place this header.
For the record: I am not opposed to placing this in media.
Regards,
Hans
> ---
> include/uapi/media/Kbuild | 1 +
> include/uapi/media/imx.h | 21 +++++++++++++++++++++
> 2 files changed, 22 insertions(+)
> create mode 100644 include/uapi/media/imx.h
>
> diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
> index aafaa5a..fa78958 100644
> --- a/include/uapi/media/Kbuild
> +++ b/include/uapi/media/Kbuild
> @@ -1 +1,2 @@
> # UAPI Header export list
> +header-y += imx.h
> diff --git a/include/uapi/media/imx.h b/include/uapi/media/imx.h
> new file mode 100644
> index 0000000..f573de4
> --- /dev/null
> +++ b/include/uapi/media/imx.h
> @@ -0,0 +1,21 @@
> +/*
> + * Copyright (c) 2014-2015 Mentor Graphics Inc.
> + * Free Software Foundation; either version 2 of the
> + * License, or (at your option) any later version
> + */
> +
> +#ifndef __UAPI_MEDIA_IMX_H__
> +#define __UAPI_MEDIA_IMX_H__
> +
> +enum imx_ctrl_id {
> + V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE + 0),
> + V4L2_CID_IMX_FIM_NUM,
> + V4L2_CID_IMX_FIM_TOLERANCE_MIN,
> + V4L2_CID_IMX_FIM_TOLERANCE_MAX,
> + V4L2_CID_IMX_FIM_NUM_SKIP,
> +};
> +
> +#endif
>
> This adds a header file for use by userspace programs wanting to interact
> with the i.MX media driver. It defines custom v4l2 controls for the
> i.MX v4l2 subdevices.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
I would not export this while the imx driver is in staging.
Also, traditionally exported media headers are in linux, not media.
I would like to have a discussion about that before deciding where to
place this header.
For the record: I am not opposed to placing this in media.
Regards,
Hans
> ---
> include/uapi/media/Kbuild | 1 +
> include/uapi/media/imx.h | 21 +++++++++++++++++++++
> 2 files changed, 22 insertions(+)
> create mode 100644 include/uapi/media/imx.h
>
> diff --git a/include/uapi/media/Kbuild b/include/uapi/media/Kbuild
> index aafaa5a..fa78958 100644
> --- a/include/uapi/media/Kbuild
> +++ b/include/uapi/media/Kbuild
> @@ -1 +1,2 @@
> # UAPI Header export list
> +header-y += imx.h
> diff --git a/include/uapi/media/imx.h b/include/uapi/media/imx.h
> new file mode 100644
> index 0000000..f573de4
> --- /dev/null
> +++ b/include/uapi/media/imx.h
> @@ -0,0 +1,21 @@
> +/*
> + * Copyright (c) 2014-2015 Mentor Graphics Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by the
> + * This program is free software; you can redistribute it and/or modify
> + * Free Software Foundation; either version 2 of the
> + * License, or (at your option) any later version
> + */
> +
> +#ifndef __UAPI_MEDIA_IMX_H__
> +#define __UAPI_MEDIA_IMX_H__
> +
> +enum imx_ctrl_id {
> + V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE + 0),
> + V4L2_CID_IMX_FIM_NUM,
> + V4L2_CID_IMX_FIM_TOLERANCE_MIN,
> + V4L2_CID_IMX_FIM_TOLERANCE_MAX,
> + V4L2_CID_IMX_FIM_NUM_SKIP,
> +};
> +
> +#endif
>
Hans Verkuil
Mar 10, 2017, 7:10:05 AM3/10/17
to
On 10/03/17 05:52, Steve Longerbeam wrote:
> Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
> output device has measured an interval between the reception or transmit
> completion of two consecutive frames of video that is outside the nominal
> frame interval by some tolerance value.
Reading back what was said on this I agree with Sakari that this doesn't
> output device has measured an interval between the reception or transmit
> completion of two consecutive frames of video that is outside the nominal
> frame interval by some tolerance value.
belong here.
Userspace can detect this just as easily (if not easier) with a timeout.
Regards,
Hans
Hans Verkuil
Mar 10, 2017, 7:10:06 AM3/10/17
to
On 10/03/17 05:52, Steve Longerbeam wrote:
> Add a NEW_FRAME_BEFORE_EOF event to signal that a video capture or
> output device has signaled a new frame is ready before a previous
> frame has completed reception or transmission. This usually indicates
> a DMA read/write channel is having trouble gaining bus access.
This too is a weird event. Based on what you describe this basically means
> output device has signaled a new frame is ready before a previous
> frame has completed reception or transmission. This usually indicates
> a DMA read/write channel is having trouble gaining bus access.
that the previous frame is incomplete, in which case you would typically
return the buffer with the V4L2_BUF_FLAG_ERROR bit set.
Using an event for this is not a good idea.
Regards,
Hans
Steve Longerbeam
Mar 10, 2017, 1:40:05 PM3/10/17
to
Hi Hans,
On 03/10/2017 04:03 AM, Hans Verkuil wrote:
> On 10/03/17 05:52, Steve Longerbeam wrote:
>> Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
>> output device has measured an interval between the reception or transmit
>> completion of two consecutive frames of video that is outside the nominal
>> frame interval by some tolerance value.
>
> Reading back what was said on this I agree with Sakari that this doesn't
> belong here.
>
> Userspace can detect this just as easily (if not easier) with a timeout.
>
Unfortunately measuring frame intervals from userland is not accurate
enough for i.MX6.
The issue here is that the IPUv3, specifically the CSI unit, can
permanently lose vertical sync if there are truncated frames sent
on the bt.656 bus. We have seen a single missing line of video cause
loss of vertical sync. The only way to correct this is to shutdown
the IPU capture hardware and restart, which can be accomplished
simply by restarting streaming from userland.
There are no other indicators from the sensor about these short
frame events (believe me, we've exhausted all avenues with the ADV718x).
And the IPUv3 DMA engine has no status indicators for short frames
either. So the only way to detect them is by measuring frame intervals.
The intervals have to be able to resolve a single line of missing video.
With a PAL video source that requires better than 58 usec accuracy.
There is too much uncertainty to resolve this at user level. The
driver is able to resolve this by measuring intervals between hardware
interrupts as long as interrupt latency is reasonably low, and we
have another method using the i.MX6 hardware input capture support
that can measure these intervals very accurately with no errors
introduced by interrupt latency.
I made this event a private event to imx-media driver in a previous
iteration, so I can return it to a private event, but this can't be
done at user level.
Steve
On 03/10/2017 04:03 AM, Hans Verkuil wrote:
> On 10/03/17 05:52, Steve Longerbeam wrote:
>> Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
>> output device has measured an interval between the reception or transmit
>> completion of two consecutive frames of video that is outside the nominal
>> frame interval by some tolerance value.
>
> Reading back what was said on this I agree with Sakari that this doesn't
> belong here.
>
> Userspace can detect this just as easily (if not easier) with a timeout.
>
enough for i.MX6.
The issue here is that the IPUv3, specifically the CSI unit, can
permanently lose vertical sync if there are truncated frames sent
on the bt.656 bus. We have seen a single missing line of video cause
loss of vertical sync. The only way to correct this is to shutdown
the IPU capture hardware and restart, which can be accomplished
simply by restarting streaming from userland.
There are no other indicators from the sensor about these short
frame events (believe me, we've exhausted all avenues with the ADV718x).
And the IPUv3 DMA engine has no status indicators for short frames
either. So the only way to detect them is by measuring frame intervals.
The intervals have to be able to resolve a single line of missing video.
With a PAL video source that requires better than 58 usec accuracy.
There is too much uncertainty to resolve this at user level. The
driver is able to resolve this by measuring intervals between hardware
interrupts as long as interrupt latency is reasonably low, and we
have another method using the i.MX6 hardware input capture support
that can measure these intervals very accurately with no errors
introduced by interrupt latency.
I made this event a private event to imx-media driver in a previous
iteration, so I can return it to a private event, but this can't be
done at user level.
Steve
Troy Kisky
Mar 10, 2017, 2:00:05 PM3/10/17
to
On 3/9/2017 8:52 PM, Steve Longerbeam wrote:
> There is a pin conflict with GPIO_6. This pin functions as a power
> input pin to the OV5642 camera sensor, but ENET uses it as the h/w
> workaround for erratum ERR006687, to wake-up the ARM cores on normal
> RX and TX packet done events. So we need to remove the h/w workaround
> to support the OV5642. The result is that the CPUidle driver will no
> longer allow entering the deep idle states on the sabrelite.
>
> This is a partial revert of
>
> commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
> interrupt.")
> commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
> for ERR006687")
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
> ---
> arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> index 8413179..89dce27 100644
> --- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> +++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> @@ -270,9 +270,6 @@
> txd1-skew-ps = <0>;
> txd2-skew-ps = <0>;
> txd3-skew-ps = <0>;
How about
> There is a pin conflict with GPIO_6. This pin functions as a power
> input pin to the OV5642 camera sensor, but ENET uses it as the h/w
> workaround for erratum ERR006687, to wake-up the ARM cores on normal
> RX and TX packet done events. So we need to remove the h/w workaround
> to support the OV5642. The result is that the CPUidle driver will no
> longer allow entering the deep idle states on the sabrelite.
>
> This is a partial revert of
>
> commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
> interrupt.")
> commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
> for ERR006687")
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
> ---
> arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> index 8413179..89dce27 100644
> --- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> +++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
> @@ -270,9 +270,6 @@
> txd1-skew-ps = <0>;
> txd2-skew-ps = <0>;
> txd3-skew-ps = <0>;
+#if !IS_ENABLED(CONFIG_VIDEO_OV5642)
> - interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>,
> - <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
> - fsl,err006687-workaround-present;
Is that allowed ?
Fabio Estevam
Mar 10, 2017, 2:20:08 PM3/10/17
to
imx6q-sabrelite-camera.dts.
This way we can keep the FEC erratum for the existing sabrelite dtb's.
Russell King - ARM Linux
Mar 10, 2017, 3:20:05 PM3/10/17
to
Version 5 gives me no v4l2 controls exposed through the video device
interface.
Just like with version 4, version 5 is completely useless with IMX219:
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
So, like v4, I can't do any further testing.
On Thu, Mar 09, 2017 at 08:52:40PM -0800, Steve Longerbeam wrote:
> In version 5:
>
> - ov5640: renamed "pwdn-gpios" to "powerdown-gpios"
>
> - ov5640: add mutex lock around the subdev op entry points.
>
> - ov5640: don't attempt to program the new mode in ov5640_set_fmt().
> Instead set a new flag, pending_mode_change, and program the new
> mode at s_stream() if flag is set.
>
> - ov5640: implement [gs]_frame_interval. As part of that, create
> ov5640_try_frame_interval(), which is used by both [gs]_frame_interval
> and [gs]_parm.
>
> - ov5640: don't attempt to set controls in ov5640_s_ctrl(), or at
> mode change, do it instead after first power-up.
>
> - video-multiplexer: include link_validate in media_entity_operations.
>
> - video-multiplexer: enforce that output pad frame interval must match
> input pad frame interval in vidsw_s_frame_interval().
>
> - video-multiplexer: initialize frame interval to a default 30 fps.
>
> - mipi csi-2: renamed "cfg" clock name property to "ref". This is the
> 27 MHz mipi csi-2 PLL reference clock.
>
> - mipi csi-2: create a hsfreq_map[] table based on
> https://community.nxp.com/docs/DOC-94312. Use it to select
> a hsfreqrange_sel value when programming the D-PHY, based on
> a max Mbps per lane. This is computed from the source subdev
> via V4L2_CID_LINK_FREQ control, and if the subdev doesn't implement
> that control, use a default hard-coded max Mbps per lane.
>
> - added required ports property description to imx-media binding doc.
>
> - removed event V4L2_EVENT_FRAME_TIMEOUT. On a frame timeout, which
> is always unrecoverable, call vb2_queue_error() instead.
>
> - export the remaining custom events to V4L2_EVENT_FRAME_INTERVAL_ERROR
> and V4L2_EVENT_NEW_FRAME_BEFORE_EOF.
>
> - vdic: use V4L2_CID_DEINTERLACING_MODE for motion compensation control
> instead of a custom control.
>
> - add v4l2_subdev_link_validate_frame_interval(). Call this in the
> link_validate imx-media subdev callbacks and video-multiplexer.
>
> - fix subdev event registration: implementation of subscribe_event()
> and unsubscribe_event() subdev ops were missing.
>
> - all calls from the pipeline to the sensor subdev have been removed.
> Only the CSI subdev still refers to a sensor, and only to retrieve
> its media bus config, which is necessary to setup the CSI interface.
>
> - add mutex locks around the imx-media subdev op entry points.
>
> - completed the propagation of all pad format parameters from sink
> pads to source pads within every imx-media subdev.
>
> - implement [gs]_frame_interval in all the imx-media subdevs.
>
> - imx-ic-prpencvf: there isn't necessarily a CSI subdev in the pipeline
> in the future, so make sure this is optional when calling the CSI's
> FIM.
>
> - the source pads that attach to capture device nodes now require the
> IPU internal pixel codes. The capture device translates these to
> v4l2 fourcc memory formats.
>
> - fix control inheritance to the capture device. When the pipeline
> was modified, the inherited controls were not being refreshed.
> v4l2_pipeline_inherit_controls() is now called only in imx-media
> link_notify() callback when a pipelink link is disabled or modified.
> imx_media_find_pipeline_video_device() is created to locate the
> capture device in the pipeline.
>
> - fix a possible race when propagating formats to the capture device.
> The subdevs and capture device use different mutex locks when setting
> formats. imx_media_capture_device_set_format() is created which acquires
> the capture device mutex when updating the capture device format.
>
> - verify all subdevs were bound in the async completion callback.
>
>
> Philipp Zabel (7):
> [media] dt-bindings: Add bindings for video-multiplexer device
> ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
> connections
> add mux and video interface bridge entity functions
> platform: add video-multiplexer subdevice driver
> media: imx: csi: fix crop rectangle changes in set_fmt
> media: imx: csi: add frame skipping support
> media: imx: csi: fix crop rectangle reset in sink set_fmt
>
> Russell King (4):
> media: imx: add support for bayer formats
> media: imx: csi: add support for bayer formats
> media: imx: mipi-csi2: enable setting and getting of frame rates
> media: imx: csi/fim: add support for frame intervals
>
> Steve Longerbeam (28):
> [media] dt-bindings: Add bindings for i.MX media driver
> [media] dt/bindings: Add bindings for OV5640
> ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
> ARM: dts: imx6qdl: add capture-subsystem device
> ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
> ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
> ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
> ARM: dts: imx6-sabreauto: create i2cmux for i2c3
> ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
> ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
> ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
> [media] v4l2: add a frame interval error event
> [media] v4l2: add a new-frame before end-of-frame event
> [media] v4l2-mc: add a function to inherit controls from a pipeline
> [media] v4l: subdev: Add function to validate frame interval
> [media] add Omnivision OV5640 sensor driver
> UAPI: Add media UAPI Kbuild file
> media: Add userspace header file for i.MX
> media: Add i.MX media core driver
> media: imx: Add Capture Device Interface
> media: imx: Add CSI subdev driver
> media: imx: Add VDIC subdev driver
> media: imx: Add IC subdev drivers
> media: imx: Add MIPI CSI-2 Receiver subdev driver
> ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
> media: imx: csi: add __csi_get_fmt
> media: imx: redo pixel format enumeration and negotiation
> media: imx: propagate sink pad formats to source pads
>
> .../devicetree/bindings/media/i2c/ov5640.txt | 45 +
> Documentation/devicetree/bindings/media/imx.txt | 74 +
> .../bindings/media/video-multiplexer.txt | 59 +
> Documentation/media/uapi/mediactl/media-types.rst | 22 +
> Documentation/media/uapi/v4l/vidioc-dqevent.rst | 12 +
> Documentation/media/v4l-drivers/imx.rst | 560 +++++
> Documentation/media/videodev2.h.rst.exceptions | 2 +
> arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 +
> arch/arm/boot/dts/imx6dl-sabresd.dts | 5 +
> arch/arm/boot/dts/imx6dl.dtsi | 185 ++
> arch/arm/boot/dts/imx6q-sabrelite.dts | 5 +
> arch/arm/boot/dts/imx6q-sabresd.dts | 5 +
> arch/arm/boot/dts/imx6q.dtsi | 121 ++
> arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 144 +-
> arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +-
> arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +-
> arch/arm/boot/dts/imx6qdl.dtsi | 17 +-
> arch/arm/configs/imx_v6_v7_defconfig | 11 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/video-multiplexer.c | 498 +++++
> drivers/media/v4l2-core/v4l2-mc.c | 48 +
> drivers/media/v4l2-core/v4l2-subdev.c | 50 +
> drivers/staging/media/Kconfig | 2 +
> drivers/staging/media/Makefile | 1 +
> drivers/staging/media/imx/Kconfig | 20 +
> drivers/staging/media/imx/Makefile | 12 +
> drivers/staging/media/imx/TODO | 17 +
> drivers/staging/media/imx/imx-ic-common.c | 113 +
> drivers/staging/media/imx/imx-ic-prp.c | 497 +++++
> drivers/staging/media/imx/imx-ic-prpencvf.c | 1236 +++++++++++
> drivers/staging/media/imx/imx-ic.h | 38 +
> drivers/staging/media/imx/imx-media-capture.c | 694 ++++++
> drivers/staging/media/imx/imx-media-csi.c | 1595 ++++++++++++++
> drivers/staging/media/imx/imx-media-dev.c | 522 +++++
> drivers/staging/media/imx/imx-media-fim.c | 463 ++++
> drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++
> drivers/staging/media/imx/imx-media-of.c | 267 +++
> drivers/staging/media/imx/imx-media-utils.c | 1009 +++++++++
> drivers/staging/media/imx/imx-media-vdic.c | 949 +++++++++
> drivers/staging/media/imx/imx-media.h | 311 +++
> drivers/staging/media/imx/imx6-mipi-csi2.c | 725 +++++++
> include/media/imx.h | 15 +
> include/media/v4l2-mc.h | 25 +
> include/media/v4l2-subdev.h | 10 +
> include/uapi/Kbuild | 1 +
> include/uapi/linux/media.h | 6 +
> include/uapi/linux/v4l2-controls.h | 4 +
> include/uapi/linux/videodev2.h | 2 +
> include/uapi/media/Kbuild | 2 +
> include/uapi/media/imx.h | 21 +
> 54 files changed, 13262 insertions(+), 27 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
> create mode 100644 Documentation/devicetree/bindings/media/imx.txt
> create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
> create mode 100644 Documentation/media/v4l-drivers/imx.rst
> create mode 100644 drivers/media/i2c/ov5640.c
> create mode 100644 drivers/media/platform/video-multiplexer.c
> create mode 100644 drivers/staging/media/imx/Kconfig
> create mode 100644 drivers/staging/media/imx/Makefile
> create mode 100644 drivers/staging/media/imx/TODO
> create mode 100644 drivers/staging/media/imx/imx-ic-common.c
> create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
> create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
> create mode 100644 drivers/staging/media/imx/imx-ic.h
> create mode 100644 drivers/staging/media/imx/imx-media-capture.c
> create mode 100644 drivers/staging/media/imx/imx-media-csi.c
> create mode 100644 drivers/staging/media/imx/imx-media-dev.c
> create mode 100644 drivers/staging/media/imx/imx-media-fim.c
> create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c
> create mode 100644 drivers/staging/media/imx/imx-media-of.c
> create mode 100644 drivers/staging/media/imx/imx-media-utils.c
> create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
> create mode 100644 drivers/staging/media/imx/imx-media.h
> create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
> create mode 100644 include/media/imx.h
> create mode 100644 include/uapi/media/Kbuild
> create mode 100644 include/uapi/media/imx.h
>
> --
> 2.7.4
>
--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
interface.
Just like with version 4, version 5 is completely useless with IMX219:
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
ipu1_csi0: pipeline start failed with -110
So, like v4, I can't do any further testing.
On Thu, Mar 09, 2017 at 08:52:40PM -0800, Steve Longerbeam wrote:
> In version 5:
>
> - ov5640: renamed "pwdn-gpios" to "powerdown-gpios"
>
> - ov5640: add mutex lock around the subdev op entry points.
>
> - ov5640: don't attempt to program the new mode in ov5640_set_fmt().
> Instead set a new flag, pending_mode_change, and program the new
> mode at s_stream() if flag is set.
>
> - ov5640: implement [gs]_frame_interval. As part of that, create
> ov5640_try_frame_interval(), which is used by both [gs]_frame_interval
> and [gs]_parm.
>
> - ov5640: don't attempt to set controls in ov5640_s_ctrl(), or at
> mode change, do it instead after first power-up.
>
> - video-multiplexer: include link_validate in media_entity_operations.
>
> - video-multiplexer: enforce that output pad frame interval must match
> input pad frame interval in vidsw_s_frame_interval().
>
> - video-multiplexer: initialize frame interval to a default 30 fps.
>
> - mipi csi-2: renamed "cfg" clock name property to "ref". This is the
> 27 MHz mipi csi-2 PLL reference clock.
>
> - mipi csi-2: create a hsfreq_map[] table based on
> https://community.nxp.com/docs/DOC-94312. Use it to select
> a hsfreqrange_sel value when programming the D-PHY, based on
> a max Mbps per lane. This is computed from the source subdev
> via V4L2_CID_LINK_FREQ control, and if the subdev doesn't implement
> that control, use a default hard-coded max Mbps per lane.
>
> - added required ports property description to imx-media binding doc.
>
> - removed event V4L2_EVENT_FRAME_TIMEOUT. On a frame timeout, which
> is always unrecoverable, call vb2_queue_error() instead.
>
> - export the remaining custom events to V4L2_EVENT_FRAME_INTERVAL_ERROR
> and V4L2_EVENT_NEW_FRAME_BEFORE_EOF.
>
> - vdic: use V4L2_CID_DEINTERLACING_MODE for motion compensation control
> instead of a custom control.
>
> - add v4l2_subdev_link_validate_frame_interval(). Call this in the
> link_validate imx-media subdev callbacks and video-multiplexer.
>
> - fix subdev event registration: implementation of subscribe_event()
> and unsubscribe_event() subdev ops were missing.
>
> - all calls from the pipeline to the sensor subdev have been removed.
> Only the CSI subdev still refers to a sensor, and only to retrieve
> its media bus config, which is necessary to setup the CSI interface.
>
> - add mutex locks around the imx-media subdev op entry points.
>
> - completed the propagation of all pad format parameters from sink
> pads to source pads within every imx-media subdev.
>
> - implement [gs]_frame_interval in all the imx-media subdevs.
>
> - imx-ic-prpencvf: there isn't necessarily a CSI subdev in the pipeline
> in the future, so make sure this is optional when calling the CSI's
> FIM.
>
> - the source pads that attach to capture device nodes now require the
> IPU internal pixel codes. The capture device translates these to
> v4l2 fourcc memory formats.
>
> - fix control inheritance to the capture device. When the pipeline
> was modified, the inherited controls were not being refreshed.
> v4l2_pipeline_inherit_controls() is now called only in imx-media
> link_notify() callback when a pipelink link is disabled or modified.
> imx_media_find_pipeline_video_device() is created to locate the
> capture device in the pipeline.
>
> - fix a possible race when propagating formats to the capture device.
> The subdevs and capture device use different mutex locks when setting
> formats. imx_media_capture_device_set_format() is created which acquires
> the capture device mutex when updating the capture device format.
>
> - verify all subdevs were bound in the async completion callback.
>
>
> Philipp Zabel (7):
> [media] dt-bindings: Add bindings for video-multiplexer device
> ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
> connections
> add mux and video interface bridge entity functions
> platform: add video-multiplexer subdevice driver
> media: imx: csi: fix crop rectangle changes in set_fmt
> media: imx: csi: add frame skipping support
> media: imx: csi: fix crop rectangle reset in sink set_fmt
>
> Russell King (4):
> media: imx: add support for bayer formats
> media: imx: csi: add support for bayer formats
> media: imx: mipi-csi2: enable setting and getting of frame rates
> media: imx: csi/fim: add support for frame intervals
>
> Steve Longerbeam (28):
> [media] dt-bindings: Add bindings for i.MX media driver
> [media] dt/bindings: Add bindings for OV5640
> ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
> ARM: dts: imx6qdl: add capture-subsystem device
> ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
> ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
> ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
> ARM: dts: imx6-sabreauto: create i2cmux for i2c3
> ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
> ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
> ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
> [media] v4l2: add a frame interval error event
> [media] v4l2: add a new-frame before end-of-frame event
> [media] v4l2-mc: add a function to inherit controls from a pipeline
> [media] v4l: subdev: Add function to validate frame interval
> [media] add Omnivision OV5640 sensor driver
> UAPI: Add media UAPI Kbuild file
> media: Add userspace header file for i.MX
> media: Add i.MX media core driver
> media: imx: Add Capture Device Interface
> media: imx: Add CSI subdev driver
> media: imx: Add VDIC subdev driver
> media: imx: Add IC subdev drivers
> media: imx: Add MIPI CSI-2 Receiver subdev driver
> ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
> media: imx: csi: add __csi_get_fmt
> media: imx: redo pixel format enumeration and negotiation
> media: imx: propagate sink pad formats to source pads
>
> .../devicetree/bindings/media/i2c/ov5640.txt | 45 +
> Documentation/devicetree/bindings/media/imx.txt | 74 +
> .../bindings/media/video-multiplexer.txt | 59 +
> Documentation/media/uapi/mediactl/media-types.rst | 22 +
> Documentation/media/uapi/v4l/vidioc-dqevent.rst | 12 +
> Documentation/media/v4l-drivers/imx.rst | 560 +++++
> Documentation/media/videodev2.h.rst.exceptions | 2 +
> arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 +
> arch/arm/boot/dts/imx6dl-sabresd.dts | 5 +
> arch/arm/boot/dts/imx6dl.dtsi | 185 ++
> arch/arm/boot/dts/imx6q-sabrelite.dts | 5 +
> arch/arm/boot/dts/imx6q-sabresd.dts | 5 +
> arch/arm/boot/dts/imx6q.dtsi | 121 ++
> arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 144 +-
> arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +-
> arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +-
> arch/arm/boot/dts/imx6qdl.dtsi | 17 +-
> arch/arm/configs/imx_v6_v7_defconfig | 11 +
> drivers/media/i2c/Kconfig | 7 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++
> drivers/media/platform/Kconfig | 8 +
> drivers/media/i2c/Makefile | 1 +
> drivers/media/i2c/ov5640.c | 2231 ++++++++++++++++++++
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/video-multiplexer.c | 498 +++++
> drivers/media/v4l2-core/v4l2-mc.c | 48 +
> drivers/media/v4l2-core/v4l2-subdev.c | 50 +
> drivers/staging/media/Kconfig | 2 +
> drivers/staging/media/Makefile | 1 +
> drivers/staging/media/imx/Kconfig | 20 +
> drivers/staging/media/imx/Makefile | 12 +
> drivers/staging/media/imx/TODO | 17 +
> drivers/staging/media/imx/imx-ic-common.c | 113 +
> drivers/staging/media/imx/imx-ic-prp.c | 497 +++++
> drivers/staging/media/imx/imx-ic-prpencvf.c | 1236 +++++++++++
> drivers/staging/media/imx/imx-ic.h | 38 +
> drivers/staging/media/imx/imx-media-capture.c | 694 ++++++
> drivers/staging/media/imx/imx-media-csi.c | 1595 ++++++++++++++
> drivers/staging/media/imx/imx-media-dev.c | 522 +++++
> drivers/staging/media/imx/imx-media-fim.c | 463 ++++
> drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++
> drivers/staging/media/imx/imx-media-of.c | 267 +++
> drivers/staging/media/imx/imx-media-utils.c | 1009 +++++++++
> drivers/staging/media/imx/imx-media-vdic.c | 949 +++++++++
> drivers/staging/media/imx/imx-media.h | 311 +++
> drivers/staging/media/imx/imx6-mipi-csi2.c | 725 +++++++
> include/media/imx.h | 15 +
> include/media/v4l2-mc.h | 25 +
> include/media/v4l2-subdev.h | 10 +
> include/uapi/Kbuild | 1 +
> include/uapi/linux/media.h | 6 +
> include/uapi/linux/v4l2-controls.h | 4 +
> include/uapi/linux/videodev2.h | 2 +
> include/uapi/media/Kbuild | 2 +
> include/uapi/media/imx.h | 21 +
> 54 files changed, 13262 insertions(+), 27 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
> create mode 100644 Documentation/devicetree/bindings/media/imx.txt
> create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
> create mode 100644 Documentation/media/v4l-drivers/imx.rst
> create mode 100644 drivers/media/i2c/ov5640.c
> create mode 100644 drivers/media/platform/video-multiplexer.c
> create mode 100644 drivers/staging/media/imx/Kconfig
> create mode 100644 drivers/staging/media/imx/Makefile
> create mode 100644 drivers/staging/media/imx/TODO
> create mode 100644 drivers/staging/media/imx/imx-ic-common.c
> create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
> create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
> create mode 100644 drivers/staging/media/imx/imx-ic.h
> create mode 100644 drivers/staging/media/imx/imx-media-capture.c
> create mode 100644 drivers/staging/media/imx/imx-media-csi.c
> create mode 100644 drivers/staging/media/imx/imx-media-dev.c
> create mode 100644 drivers/staging/media/imx/imx-media-fim.c
> create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c
> create mode 100644 drivers/staging/media/imx/imx-media-of.c
> create mode 100644 drivers/staging/media/imx/imx-media-utils.c
> create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
> create mode 100644 drivers/staging/media/imx/imx-media.h
> create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
> create mode 100644 include/media/imx.h
> create mode 100644 include/uapi/media/Kbuild
> create mode 100644 include/uapi/media/imx.h
>
> --
> 2.7.4
>
--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
Pavel Machek
Mar 10, 2017, 5:00:10 PM3/10/17
to
> Or maybe just create a new device tree for using the camera, like
> imx6q-sabrelite-camera.dts.
able to ship board with working device tree...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Fabio Estevam
Mar 10, 2017, 5:10:05 PM3/10/17
to
On Fri, Mar 10, 2017 at 6:57 PM, Pavel Machek <pa...@ucw.cz> wrote:
> And it should not depend on configuration. Hardware vendor should be
> able to ship board with working device tree...
We are talking about pin conflict here. Please read the commit log of
> And it should not depend on configuration. Hardware vendor should be
> able to ship board with working device tree...
this patch for details.
Steve Longerbeam
Mar 10, 2017, 6:30:04 PM3/10/17
to
On 03/10/2017 12:13 PM, Russell King - ARM Linux wrote:
> Version 5 gives me no v4l2 controls exposed through the video device
> interface.
>
> Just like with version 4, version 5 is completely useless with IMX219:
>
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
>
> So, like v4, I can't do any further testing.
>
from s_power(ON)?
I realize that probably means bringing the chip up to a
completely operational state and then setting it to stream
OFF in the s_power() op.
The same had to be done for the OV5640.
Steve
Steve Longerbeam
Mar 10, 2017, 6:50:05 PM3/10/17
to
latency less than 10 usec under normal system load. False events can be
generated if the latency gets bad, it's true, and that's why there is
the imx6 timer input capture approach.
Steve
Hans Verkuil
Mar 11, 2017, 6:50:04 AM3/11/17
to
see it. But if you lose vsyncs, then you never capture another frame,
right? So userspace can detect that (i.e. no new frames arrive) and
it can timeout on that. Or you detect it in the driver and restart there,
or call vb2_queue_error().
Anything really as long as this event isn't user-visible :-)
Regards,
Hans
Sakari Ailus
Mar 11, 2017, 8:50:04 AM3/11/17
to
Hi Steve,
On Thu, Mar 09, 2017 at 08:52:58PM -0800, Steve Longerbeam wrote:
> If the pads on both sides of a link specify a frame interval, then
> those frame intervals should match. Create the exported function
> v4l2_subdev_link_validate_frame_interval() to verify this. This
> function can be called in a subdevice's media_entity_operations
> or v4l2_subdev_pad_ops link_validate callbacks.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
If your only goal is to configure frame dropping on a sub-device, I suggest
to implement s_frame_interval() on the pads of that sub-device only. The
frames are then dropped according to the configured frame rates between the
sink and source pads. Say, configuring sink for 1/30 s and source 1/15 would
drop half of the incoming frames.
Considering that supporting specific frame interval on most sub-devices adds
no value or is not the interface through which it the frame rate configured,
I think it is overkill to change the link validation to expect otherwise.
--
Kind regards,
Sakari Ailus
e-mail: sakari...@iki.fi XMPP: sai...@retiisi.org.uk
On Thu, Mar 09, 2017 at 08:52:58PM -0800, Steve Longerbeam wrote:
> If the pads on both sides of a link specify a frame interval, then
> those frame intervals should match. Create the exported function
> v4l2_subdev_link_validate_frame_interval() to verify this. This
> function can be called in a subdevice's media_entity_operations
> or v4l2_subdev_pad_ops link_validate callbacks.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
to implement s_frame_interval() on the pads of that sub-device only. The
frames are then dropped according to the configured frame rates between the
sink and source pads. Say, configuring sink for 1/30 s and source 1/15 would
drop half of the incoming frames.
Considering that supporting specific frame interval on most sub-devices adds
no value or is not the interface through which it the frame rate configured,
I think it is overkill to change the link validation to expect otherwise.
--
Kind regards,
Sakari Ailus
e-mail: sakari...@iki.fi XMPP: sai...@retiisi.org.uk
Sakari Ailus
Mar 11, 2017, 8:50:05 AM3/11/17
to
Hi Steve,
On Thu, Mar 09, 2017 at 08:53:01PM -0800, Steve Longerbeam wrote:
> Add an empty UAPI Kbuild file for media UAPI headers.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
The existing V4L2 UAPI headers are under include/uapi/linux. Could you use
that directory instead?
I actually wouldn't really object doing this but it should have been done in
2002 or so when the first V4L2 header was added. Now the benefit is
questionable.
--
On Thu, Mar 09, 2017 at 08:53:01PM -0800, Steve Longerbeam wrote:
> Add an empty UAPI Kbuild file for media UAPI headers.
>
> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
The existing V4L2 UAPI headers are under include/uapi/linux. Could you use
that directory instead?
I actually wouldn't really object doing this but it should have been done in
2002 or so when the first V4L2 header was added. Now the benefit is
questionable.
--
Steve Longerbeam
Mar 11, 2017, 1:20:04 PM3/11/17
to
captures portions of two different frames, resulting in a permanent
"split image", with one frame containing portions of two consecutive
images. Or, the video rolls continuously, if you remember the old CRT
television sets of yore, it's the same rolling effect.
> So userspace can detect that (i.e. no new frames arrive) and
> it can timeout on that. Or you detect it in the driver and restart there,
> or call vb2_queue_error().
>
or rolling.
> Anything really as long as this event isn't user-visible :-)
the error, and can't correct it by stream restart.
Steve
Steve Longerbeam
Mar 11, 2017, 1:30:05 PM3/11/17
to
On 03/11/2017 05:49 AM, Sakari Ailus wrote:
> Hi Steve,
>
> On Thu, Mar 09, 2017 at 08:53:01PM -0800, Steve Longerbeam wrote:
>> Add an empty UAPI Kbuild file for media UAPI headers.
>>
>> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
>
> The existing V4L2 UAPI headers are under include/uapi/linux. Could you use
> that directory instead?
>
> I actually wouldn't really object doing this but it should have been done in
> 2002 or so when the first V4L2 header was added. Now the benefit is
> questionable.
>
eventually, but for now I'll go ahead and put under uapi/linux.
Steve
Steve Longerbeam
Mar 11, 2017, 2:00:05 PM3/11/17
to
On 03/11/2017 10:51 AM, Russell King - ARM Linux wrote:
> On Sat, Mar 11, 2017 at 10:14:49AM -0800, Steve Longerbeam wrote:
>> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>>> It's fine to use an internal event as long as the end-user doesn't
>>> see it. But if you lose vsyncs, then you never capture another frame,
>>> right?
>>
>> No, that's not correct. By loss of vertical sync, I mean the IPU
>> captures portions of two different frames, resulting in a permanent
>> "split image", with one frame containing portions of two consecutive
>> images. Or, the video rolls continuously, if you remember the old CRT
>> television sets of yore, it's the same rolling effect.
>
> I have seen that rolling effect, but the iMX6 regains correct sync
>> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>>> It's fine to use an internal event as long as the end-user doesn't
>>> see it. But if you lose vsyncs, then you never capture another frame,
>>> right?
>>
>> No, that's not correct. By loss of vertical sync, I mean the IPU
>> captures portions of two different frames, resulting in a permanent
>> "split image", with one frame containing portions of two consecutive
>> images. Or, the video rolls continuously, if you remember the old CRT
>> television sets of yore, it's the same rolling effect.
>
> within one complete "roll" just fine here with IMX219. However, it
> has always recovered.
I've seen permanent split images, and rolling that continues for
minutes.
>
> So, I don't think there's a problem with the iMX6 part of the
> processing, and so I don't think we should cripple the iMX6 capture
> drivers for this problem.
>
> It seems to me that the problem is with the source.
The the problem is almost surely in the CSI. It's handling of the
bt.656 sync codes is broken in some way.
Steve
Russell King - ARM Linux
Mar 11, 2017, 2:00:05 PM3/11/17
to
On Sat, Mar 11, 2017 at 10:14:49AM -0800, Steve Longerbeam wrote:
> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
> >It's fine to use an internal event as long as the end-user doesn't
> >see it. But if you lose vsyncs, then you never capture another frame,
> >right?
>
> No, that's not correct. By loss of vertical sync, I mean the IPU
> captures portions of two different frames, resulting in a permanent
> "split image", with one frame containing portions of two consecutive
> images. Or, the video rolls continuously, if you remember the old CRT
> television sets of yore, it's the same rolling effect.
> >It's fine to use an internal event as long as the end-user doesn't
> >see it. But if you lose vsyncs, then you never capture another frame,
> >right?
>
> No, that's not correct. By loss of vertical sync, I mean the IPU
> captures portions of two different frames, resulting in a permanent
> "split image", with one frame containing portions of two consecutive
> images. Or, the video rolls continuously, if you remember the old CRT
> television sets of yore, it's the same rolling effect.
I have seen that rolling effect, but the iMX6 regains correct sync
within one complete "roll" just fine here with IMX219. However, it
has always recovered.
within one complete "roll" just fine here with IMX219. However, it
has always recovered.
So, I don't think there's a problem with the iMX6 part of the
processing, and so I don't think we should cripple the iMX6 capture
drivers for this problem.
It seems to me that the problem is with the source.
processing, and so I don't think we should cripple the iMX6 capture
drivers for this problem.
It seems to me that the problem is with the source.
Steve Longerbeam
Mar 11, 2017, 2:10:05 PM3/11/17
to
On 03/11/2017 10:51 AM, Russell King - ARM Linux wrote:
> On Sat, Mar 11, 2017 at 10:14:49AM -0800, Steve Longerbeam wrote:
>> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>>> It's fine to use an internal event as long as the end-user doesn't
>>> see it. But if you lose vsyncs, then you never capture another frame,
>>> right?
>>
>> No, that's not correct. By loss of vertical sync, I mean the IPU
>> captures portions of two different frames, resulting in a permanent
>> "split image", with one frame containing portions of two consecutive
>> images. Or, the video rolls continuously, if you remember the old CRT
>> television sets of yore, it's the same rolling effect.
>
> I have seen that rolling effect, but the iMX6 regains correct sync
> within one complete "roll" just fine here with IMX219. However, it
> has always recovered.
>
> So, I don't think there's a problem with the iMX6 part of the
> processing, and so I don't think we should cripple the iMX6 capture
> drivers for this problem.
And there is no crippling going on. Measuring the frame intervals
>> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>>> It's fine to use an internal event as long as the end-user doesn't
>>> see it. But if you lose vsyncs, then you never capture another frame,
>>> right?
>>
>> No, that's not correct. By loss of vertical sync, I mean the IPU
>> captures portions of two different frames, resulting in a permanent
>> "split image", with one frame containing portions of two consecutive
>> images. Or, the video rolls continuously, if you remember the old CRT
>> television sets of yore, it's the same rolling effect.
>
> I have seen that rolling effect, but the iMX6 regains correct sync
> within one complete "roll" just fine here with IMX219. However, it
> has always recovered.
>
> So, I don't think there's a problem with the iMX6 part of the
> processing, and so I don't think we should cripple the iMX6 capture
> drivers for this problem.
adds very little overhead.
Steve
Steve Longerbeam
Mar 11, 2017, 3:40:05 PM3/11/17
to
future, I do agree with you that a subdev that has no control over
frame rate has no business implementing the get|set ops.
In the imx-media subdevs, the only one that can affect frame rate (via
frame skipping) is the CSI. So I'll go ahead and remove the
[gs]_frame_interval ops from the others. I can remove this patch as
a result.
Steve
Steve Longerbeam
Mar 11, 2017, 7:40:04 PM3/11/17
to
On 03/10/2017 12:13 PM, Russell King - ARM Linux wrote:
> Version 5 gives me no v4l2 controls exposed through the video device
> interface.
>
> Just like with version 4, version 5 is completely useless with IMX219:
>
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
> imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
> ipu1_csi0: pipeline start failed with -110
>
LP-11 state on power on, perhaps the csi-2 receiver can be a little
more lenient on the transmitter and make the LP-11 timeout a warning
instead of error-out.
Can you try the attached change on top of the version 5 patchset?
If that doesn't work then you're just going to have to fix the bug
in imx219.
Steve
Russell King - ARM Linux
Mar 12, 2017, 1:50:04 PM3/12/17
to
it supports? That's likely to be over the maximum data rate specified
for iMX6Q if it's wired up using four lanes.
Also, as I've already said, I think that powering on the sensor just
because it's got an enabled media-controller link is a silly idea.
Right now, the only way of using the imx6 capture stuff is to manually
configure it with media-ctl, which means that happens either at boot
due to a custom boot script, or when you first use it (by manually
running a script.)
This results in the sensor staying powered from that point onwards,
wasting power unnecessarily.
Russell King - ARM Linux
Mar 12, 2017, 2:00:06 PM3/12/17
to
I've just looked at my test system's dmesg, and spotted this in the log.
It's been a while since these popped out of the kernel, so I don't know
what caused them (other than the obvious, a media-ctl command.)
My script which sets this up only enables links, and then configures the
formats etc, and doesn't disable them, so I don't see why the power
count should be going negative.
------------[ cut here ]------------
WARNING: CPU: 1 PID: 1889 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0x9c/0xa8 [imx_media_csi]
Modules linked in: caam_jr uvcvideo snd_soc_imx_sgtl5000 snd_soc_fsl_asoc_card snd_soc_imx_spdif imx_media_csi(C) imx6_mipi_csi2(C) snd_soc_imx_audmux snd_soc_sgtl5000 imx219 imx_media_ic(C) imx_media_capture(C) imx_media_vdic(C) caam video_multiplexer imx_sdma coda v4l2_mem2mem videobuf2_v4l2 imx2_wdt imx_vdoa videobuf2_dma_contig videobuf2_core videobuf2_vmalloc videobuf2_memops snd_soc_fsl_ssi
imx_thermal snd_soc_fsl_spdif imx_pcm_dma imx_media(C) imx_media_common(C) nfsd
rc_pinnacle_pctv_hd dw_hdmi_ahb_audio dw_hdmi_cec etnaviv
CPU: 1 PID: 1889 Comm: media-ctl Tainted: G C 4.11.0-rc1+ #2125
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<c0013ba4>] (dump_backtrace) from [<c0013de4>] (show_stack+0x18/0x1c)
r6:600e0013 r5:ffffffff r4:00000000 r3:00000000
[<c0013dcc>] (show_stack) from [<c033f728>] (dump_stack+0xa4/0xdc)
[<c033f684>] (dump_stack) from [<c00326cc>] (__warn+0xdc/0x108)
r6:bf124014 r5:00000000 r4:00000000 r3:c09ea4a8
[<c00325f0>] (__warn) from [<c0032720>] (warn_slowpath_null+0x28/0x30)
r10:ede00010 r8:ede4a348 r7:d039501c r6:d0395140 r5:00000000 r4:d0395010
[<c00326f8>] (warn_slowpath_null) from [<bf1229e0>] (csi_s_power+0x9c/0xa8 [imx_media_csi])
[<bf122944>] (csi_s_power [imx_media_csi]) from [<bf04a930>] (imx_media_set_power+0x3c/0x108 [imx_media_common])
r7:d039501c r6:00000000 r5:00000000 r4:0000000c
[<bf04a8f4>] (imx_media_set_power [imx_media_common]) from [<bf04aa34>] (imx_media_pipeline_set_power+0x38/0x40 [imx_media_common])
r10:00000001 r9:00000001 r8:ede4a348 r7:ede00010 r6:ede4a348 r5:d039501c
r4:00000001
[<bf04a9fc>] (imx_media_pipeline_set_power [imx_media_common]) from [<bf052148>] (imx_media_link_notify+0xf0/0x144 [imx_media])
r7:ede00010 r6:ed59f900 r5:00000000 r4:d039501c
[<bf052058>] (imx_media_link_notify [imx_media]) from [<c04fa858>] (__media_entity_setup_link+0x110/0x1d8)
r10:c0347c03 r9:d7eb3dc8 r8:befe92b0 r7:ede00010 r6:00000000 r5:00000001
r4:ed59f900 r3:bf052058
[<c04fa748>] (__media_entity_setup_link) from [<c04f9bb0>] (media_device_setup_link+0x84/0x90)
r7:ede00010 r6:ede00010 r5:ef3fd810 r4:d7eb3dc8
[<c04f9b2c>] (media_device_setup_link) from [<c04f9e94>] (media_device_ioctl+0xa4/0x148)
r6:00000000 r5:d7eb3dc8 r4:c077b014 r3:c04f9b2c
[<c04f9df0>] (media_device_ioctl) from [<c04fa3a0>] (media_ioctl+0x38/0x4c)
r10:ed5eca68 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:00000003 r5:e82ca280
r4:c0190304
[<c04fa368>] (media_ioctl) from [<c018f9c0>] (do_vfs_ioctl+0x98/0x9a0)
[<c018f928>] (do_vfs_ioctl) from [<c0190304>] (SyS_ioctl+0x3c/0x60)
r10:00000000 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:c0347c03 r5:e82ca280
r4:e82ca280
[<c01902c8>] (SyS_ioctl) from [<c000fd60>] (ret_fast_syscall+0x0/0x1c)
r8:c000ff04 r7:00000036 r6:000261d0 r5:00000001 r4:009162e8 r3:00000001
---[ end trace 4fdd40e5adfc4485 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 1889 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0x9c/0xa8 [imx_media_csi]
Modules linked in: caam_jr uvcvideo snd_soc_imx_sgtl5000 snd_soc_fsl_asoc_card snd_soc_imx_spdif imx_media_csi(C) imx6_mipi_csi2(C) snd_soc_imx_audmux snd_soc_sgtl5000 imx219 imx_media_ic(C) imx_media_capture(C) imx_media_vdic(C) caam video_multiplexer imx_sdma coda v4l2_mem2mem videobuf2_v4l2 imx2_wdt imx_vdoa videobuf2_dma_contig videobuf2_core videobuf2_vmalloc videobuf2_memops snd_soc_fsl_ssi
imx_thermal snd_soc_fsl_spdif imx_pcm_dma imx_media(C) imx_media_common(C) nfsd
rc_pinnacle_pctv_hd dw_hdmi_ahb_audio dw_hdmi_cec etnaviv
CPU: 1 PID: 1889 Comm: media-ctl Tainted: G WC 4.11.0-rc1+ #2125
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<c0013ba4>] (dump_backtrace) from [<c0013de4>] (show_stack+0x18/0x1c)
r6:600e0013 r5:ffffffff r4:00000000 r3:00000000
[<c0013dcc>] (show_stack) from [<c033f728>] (dump_stack+0xa4/0xdc)
[<c033f684>] (dump_stack) from [<c00326cc>] (__warn+0xdc/0x108)
r6:bf124014 r5:00000000 r4:00000000 r3:c09ea4a8
[<c00325f0>] (__warn) from [<c0032720>] (warn_slowpath_null+0x28/0x30)
r10:ede00010 r8:ede4a348 r7:ee000800 r6:d0395140 r5:00000000 r4:d0395010
[<c00326f8>] (warn_slowpath_null) from [<bf1229e0>] (csi_s_power+0x9c/0xa8 [imx_media_csi])
[<bf122944>] (csi_s_power [imx_media_csi]) from [<bf04a930>] (imx_media_set_power+0x3c/0x108 [imx_media_common])
r7:ee000800 r6:00000000 r5:00000000 r4:0000000c
[<bf04a8f4>] (imx_media_set_power [imx_media_common]) from [<bf04aa34>] (imx_media_pipeline_set_power+0x38/0x40 [imx_media_common])
r10:00000001 r9:00000001 r8:ede4a348 r7:ede00010 r6:ede4a348 r5:ee000800
r4:00000001
[<bf04a9fc>] (imx_media_pipeline_set_power [imx_media_common]) from [<bf052148>] (imx_media_link_notify+0xf0/0x144 [imx_media])
r7:ede00010 r6:d0320480 r5:ee000800 r4:ee000800
[<bf052058>] (imx_media_link_notify [imx_media]) from [<c04fa858>] (__media_entity_setup_link+0x110/0x1d8)
r10:c0347c03 r9:d7eb3dc8 r8:befe92b0 r7:ede00010 r6:00000000 r5:00000001
r4:d0320480 r3:bf052058
[<c04fa748>] (__media_entity_setup_link) from [<c04f9bb0>] (media_device_setup_link+0x84/0x90)
r7:ede00010 r6:ede00010 r5:d039501c r4:d7eb3dc8
[<c04f9b2c>] (media_device_setup_link) from [<c04f9e94>] (media_device_ioctl+0xa4/0x148)
r6:00000000 r5:d7eb3dc8 r4:c077b014 r3:c04f9b2c
[<c04f9df0>] (media_device_ioctl) from [<c04fa3a0>] (media_ioctl+0x38/0x4c)
r10:ed5eca68 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:00000003 r5:e82ca500
r4:c0190304
[<c04fa368>] (media_ioctl) from [<c018f9c0>] (do_vfs_ioctl+0x98/0x9a0)
[<c018f928>] (do_vfs_ioctl) from [<c0190304>] (SyS_ioctl+0x3c/0x60)
r10:00000000 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:c0347c03 r5:e82ca500
r4:e82ca500
[<c01902c8>] (SyS_ioctl) from [<c000fd60>] (ret_fast_syscall+0x0/0x1c)
r8:c000ff04 r7:00000036 r6:000261d0 r5:00000001 r4:0091737c r3:00000001
---[ end trace 4fdd40e5adfc4486 ]---
It's been a while since these popped out of the kernel, so I don't know
what caused them (other than the obvious, a media-ctl command.)
My script which sets this up only enables links, and then configures the
formats etc, and doesn't disable them, so I don't see why the power
count should be going negative.
------------[ cut here ]------------
WARNING: CPU: 1 PID: 1889 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0x9c/0xa8 [imx_media_csi]
Modules linked in: caam_jr uvcvideo snd_soc_imx_sgtl5000 snd_soc_fsl_asoc_card snd_soc_imx_spdif imx_media_csi(C) imx6_mipi_csi2(C) snd_soc_imx_audmux snd_soc_sgtl5000 imx219 imx_media_ic(C) imx_media_capture(C) imx_media_vdic(C) caam video_multiplexer imx_sdma coda v4l2_mem2mem videobuf2_v4l2 imx2_wdt imx_vdoa videobuf2_dma_contig videobuf2_core videobuf2_vmalloc videobuf2_memops snd_soc_fsl_ssi
imx_thermal snd_soc_fsl_spdif imx_pcm_dma imx_media(C) imx_media_common(C) nfsd
rc_pinnacle_pctv_hd dw_hdmi_ahb_audio dw_hdmi_cec etnaviv
CPU: 1 PID: 1889 Comm: media-ctl Tainted: G C 4.11.0-rc1+ #2125
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<c0013ba4>] (dump_backtrace) from [<c0013de4>] (show_stack+0x18/0x1c)
r6:600e0013 r5:ffffffff r4:00000000 r3:00000000
[<c0013dcc>] (show_stack) from [<c033f728>] (dump_stack+0xa4/0xdc)
[<c033f684>] (dump_stack) from [<c00326cc>] (__warn+0xdc/0x108)
r6:bf124014 r5:00000000 r4:00000000 r3:c09ea4a8
[<c00325f0>] (__warn) from [<c0032720>] (warn_slowpath_null+0x28/0x30)
r10:ede00010 r8:ede4a348 r7:d039501c r6:d0395140 r5:00000000 r4:d0395010
[<c00326f8>] (warn_slowpath_null) from [<bf1229e0>] (csi_s_power+0x9c/0xa8 [imx_media_csi])
[<bf122944>] (csi_s_power [imx_media_csi]) from [<bf04a930>] (imx_media_set_power+0x3c/0x108 [imx_media_common])
r7:d039501c r6:00000000 r5:00000000 r4:0000000c
[<bf04a8f4>] (imx_media_set_power [imx_media_common]) from [<bf04aa34>] (imx_media_pipeline_set_power+0x38/0x40 [imx_media_common])
r10:00000001 r9:00000001 r8:ede4a348 r7:ede00010 r6:ede4a348 r5:d039501c
r4:00000001
[<bf04a9fc>] (imx_media_pipeline_set_power [imx_media_common]) from [<bf052148>] (imx_media_link_notify+0xf0/0x144 [imx_media])
r7:ede00010 r6:ed59f900 r5:00000000 r4:d039501c
[<bf052058>] (imx_media_link_notify [imx_media]) from [<c04fa858>] (__media_entity_setup_link+0x110/0x1d8)
r10:c0347c03 r9:d7eb3dc8 r8:befe92b0 r7:ede00010 r6:00000000 r5:00000001
r4:ed59f900 r3:bf052058
[<c04fa748>] (__media_entity_setup_link) from [<c04f9bb0>] (media_device_setup_link+0x84/0x90)
r7:ede00010 r6:ede00010 r5:ef3fd810 r4:d7eb3dc8
[<c04f9b2c>] (media_device_setup_link) from [<c04f9e94>] (media_device_ioctl+0xa4/0x148)
r6:00000000 r5:d7eb3dc8 r4:c077b014 r3:c04f9b2c
[<c04f9df0>] (media_device_ioctl) from [<c04fa3a0>] (media_ioctl+0x38/0x4c)
r10:ed5eca68 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:00000003 r5:e82ca280
r4:c0190304
[<c04fa368>] (media_ioctl) from [<c018f9c0>] (do_vfs_ioctl+0x98/0x9a0)
[<c018f928>] (do_vfs_ioctl) from [<c0190304>] (SyS_ioctl+0x3c/0x60)
r10:00000000 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:c0347c03 r5:e82ca280
r4:e82ca280
[<c01902c8>] (SyS_ioctl) from [<c000fd60>] (ret_fast_syscall+0x0/0x1c)
r8:c000ff04 r7:00000036 r6:000261d0 r5:00000001 r4:009162e8 r3:00000001
---[ end trace 4fdd40e5adfc4485 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 1889 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0x9c/0xa8 [imx_media_csi]
Modules linked in: caam_jr uvcvideo snd_soc_imx_sgtl5000 snd_soc_fsl_asoc_card snd_soc_imx_spdif imx_media_csi(C) imx6_mipi_csi2(C) snd_soc_imx_audmux snd_soc_sgtl5000 imx219 imx_media_ic(C) imx_media_capture(C) imx_media_vdic(C) caam video_multiplexer imx_sdma coda v4l2_mem2mem videobuf2_v4l2 imx2_wdt imx_vdoa videobuf2_dma_contig videobuf2_core videobuf2_vmalloc videobuf2_memops snd_soc_fsl_ssi
imx_thermal snd_soc_fsl_spdif imx_pcm_dma imx_media(C) imx_media_common(C) nfsd
rc_pinnacle_pctv_hd dw_hdmi_ahb_audio dw_hdmi_cec etnaviv
CPU: 1 PID: 1889 Comm: media-ctl Tainted: G WC 4.11.0-rc1+ #2125
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<c0013ba4>] (dump_backtrace) from [<c0013de4>] (show_stack+0x18/0x1c)
r6:600e0013 r5:ffffffff r4:00000000 r3:00000000
[<c0013dcc>] (show_stack) from [<c033f728>] (dump_stack+0xa4/0xdc)
[<c033f684>] (dump_stack) from [<c00326cc>] (__warn+0xdc/0x108)
r6:bf124014 r5:00000000 r4:00000000 r3:c09ea4a8
[<c00325f0>] (__warn) from [<c0032720>] (warn_slowpath_null+0x28/0x30)
r10:ede00010 r8:ede4a348 r7:ee000800 r6:d0395140 r5:00000000 r4:d0395010
[<c00326f8>] (warn_slowpath_null) from [<bf1229e0>] (csi_s_power+0x9c/0xa8 [imx_media_csi])
[<bf122944>] (csi_s_power [imx_media_csi]) from [<bf04a930>] (imx_media_set_power+0x3c/0x108 [imx_media_common])
r7:ee000800 r6:00000000 r5:00000000 r4:0000000c
[<bf04a8f4>] (imx_media_set_power [imx_media_common]) from [<bf04aa34>] (imx_media_pipeline_set_power+0x38/0x40 [imx_media_common])
r10:00000001 r9:00000001 r8:ede4a348 r7:ede00010 r6:ede4a348 r5:ee000800
r4:00000001
[<bf04a9fc>] (imx_media_pipeline_set_power [imx_media_common]) from [<bf052148>] (imx_media_link_notify+0xf0/0x144 [imx_media])
r7:ede00010 r6:d0320480 r5:ee000800 r4:ee000800
[<bf052058>] (imx_media_link_notify [imx_media]) from [<c04fa858>] (__media_entity_setup_link+0x110/0x1d8)
r10:c0347c03 r9:d7eb3dc8 r8:befe92b0 r7:ede00010 r6:00000000 r5:00000001
r4:d0320480 r3:bf052058
[<c04fa748>] (__media_entity_setup_link) from [<c04f9bb0>] (media_device_setup_link+0x84/0x90)
r7:ede00010 r6:ede00010 r5:d039501c r4:d7eb3dc8
[<c04f9b2c>] (media_device_setup_link) from [<c04f9e94>] (media_device_ioctl+0xa4/0x148)
r6:00000000 r5:d7eb3dc8 r4:c077b014 r3:c04f9b2c
[<c04f9df0>] (media_device_ioctl) from [<c04fa3a0>] (media_ioctl+0x38/0x4c)
r10:ed5eca68 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:00000003 r5:e82ca500
r4:c0190304
[<c04fa368>] (media_ioctl) from [<c018f9c0>] (do_vfs_ioctl+0x98/0x9a0)
[<c018f928>] (do_vfs_ioctl) from [<c0190304>] (SyS_ioctl+0x3c/0x60)
r10:00000000 r9:d7eb2000 r8:befe92b0 r7:00000003 r6:c0347c03 r5:e82ca500
r4:e82ca500
[<c01902c8>] (SyS_ioctl) from [<c000fd60>] (ret_fast_syscall+0x0/0x1c)
r8:c000ff04 r7:00000036 r6:000261d0 r5:00000001 r4:0091737c r3:00000001
---[ end trace 4fdd40e5adfc4486 ]---
Steve Longerbeam
Mar 12, 2017, 3:30:04 PM3/12/17
to
On 03/12/2017 10:51 AM, Russell King - ARM Linux wrote:
> I've just looked at my test system's dmesg, and spotted this in the log.
> It's been a while since these popped out of the kernel, so I don't know
> what caused them (other than the obvious, a media-ctl command.)
>
> My script which sets this up only enables links, and then configures the
> formats etc, and doesn't disable them, so I don't see why the power
> count should be going negative.
multiple times, and imx_media_link_notify() complies, and so
csi_s_power(OFF) gets called multiple times, and so that WARN_ON()
in there is silly, I borrowed this from other MC driver examples,
but it makes no sense to me, I'll remove it and prevent the power
count from going negative.
Steve
Russell King - ARM Linux
Mar 12, 2017, 3:40:05 PM3/12/17
to
On Sun, Mar 12, 2017 at 12:21:45PM -0700, Steve Longerbeam wrote:
> There's actually nothing preventing userland from disabling a link
> multiple times, and imx_media_link_notify() complies, and so
> csi_s_power(OFF) gets called multiple times, and so that WARN_ON()
> in there is silly, I borrowed this from other MC driver examples,
> but it makes no sense to me, I'll remove it and prevent the power
> count from going negative.
Hmm. So what happens if one of the CSI's links is enabled, and we
> There's actually nothing preventing userland from disabling a link
> multiple times, and imx_media_link_notify() complies, and so
> csi_s_power(OFF) gets called multiple times, and so that WARN_ON()
> in there is silly, I borrowed this from other MC driver examples,
> but it makes no sense to me, I'll remove it and prevent the power
> count from going negative.
disable a different link from the CSI several times? Doesn't that
mean the power count will go to zero despite there being an enabled
link?
Russell King - ARM Linux
Mar 12, 2017, 3:50:05 PM3/12/17
to
Another issue.
The "reboot and the /dev/video* devices come up in a completely
different order" problem seems to exist with this version.
The dot graph I supplied previously had "ipu1_csi0 capture" on
/dev/video4. I've just rebooted, and now I find it's on
/dev/video2 instead.
Here's the extract from the .dot file of the old listing:
n00000018 [label="ipu1_ic_prpenc capture\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
n00000021 [label="ipu1_ic_prpvf capture\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
n0000002e [label="ipu2_ic_prpenc capture\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
n00000037 [label="ipu2_ic_prpvf capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
n00000048 [label="ipu1_csi0 capture\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
n00000052 [label="ipu1_csi1 capture\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
n00000062 [label="ipu2_csi0 capture\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
n0000006c [label="ipu2_csi1 capture\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
and here's the same after reboot:
n00000014 [label="ipu1_csi0 capture\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
n0000001e [label="ipu1_csi1 capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
n00000028 [label="ipu2_csi0 capture\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
n00000035 [label="ipu1_ic_prpenc capture\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
n0000003e [label="ipu1_ic_prpvf capture\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
n0000004c [label="ipu2_csi1 capture\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
n00000059 [label="ipu2_ic_prpenc capture\n/dev/video8", shape=box, style=filled, fillcolor=yellow]
n00000062 [label="ipu2_ic_prpvf capture\n/dev/video9", shape=box, style=filled, fillcolor=yellow]
(/dev/video0 and /dev/video1 are taken up by CODA, since I updated the
names of the firmware files, and now CODA initialises... seems the
back-compat filenames don't work, but that's not a problem with imx6
capture.)
The "reboot and the /dev/video* devices come up in a completely
different order" problem seems to exist with this version.
The dot graph I supplied previously had "ipu1_csi0 capture" on
/dev/video4. I've just rebooted, and now I find it's on
/dev/video2 instead.
Here's the extract from the .dot file of the old listing:
n00000018 [label="ipu1_ic_prpenc capture\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
n00000021 [label="ipu1_ic_prpvf capture\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
n0000002e [label="ipu2_ic_prpenc capture\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
n00000037 [label="ipu2_ic_prpvf capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
n00000048 [label="ipu1_csi0 capture\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
n00000052 [label="ipu1_csi1 capture\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
n00000062 [label="ipu2_csi0 capture\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
n0000006c [label="ipu2_csi1 capture\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
and here's the same after reboot:
n00000014 [label="ipu1_csi0 capture\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
n0000001e [label="ipu1_csi1 capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
n00000028 [label="ipu2_csi0 capture\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
n00000035 [label="ipu1_ic_prpenc capture\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
n0000003e [label="ipu1_ic_prpvf capture\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
n0000004c [label="ipu2_csi1 capture\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
n00000059 [label="ipu2_ic_prpenc capture\n/dev/video8", shape=box, style=filled, fillcolor=yellow]
n00000062 [label="ipu2_ic_prpvf capture\n/dev/video9", shape=box, style=filled, fillcolor=yellow]
(/dev/video0 and /dev/video1 are taken up by CODA, since I updated the
names of the firmware files, and now CODA initialises... seems the
back-compat filenames don't work, but that's not a problem with imx6
capture.)
Steve Longerbeam
Mar 12, 2017, 3:50:06 PM3/12/17
to
On 03/12/2017 12:29 PM, Russell King - ARM Linux wrote:
> On Sun, Mar 12, 2017 at 12:21:45PM -0700, Steve Longerbeam wrote:
>> There's actually nothing preventing userland from disabling a link
>> multiple times, and imx_media_link_notify() complies, and so
>> csi_s_power(OFF) gets called multiple times, and so that WARN_ON()
>> in there is silly, I borrowed this from other MC driver examples,
>> but it makes no sense to me, I'll remove it and prevent the power
>> count from going negative.
>
> Hmm. So what happens if one of the CSI's links is enabled, and we
> disable a different link from the CSI several times? Doesn't that
> mean the power count will go to zero despite there being an enabled
> link?
But one of its other links has been disabled, meaning the pipeline as
a whole is disabled. So I think it makes sense to power down the CSI,
the pipeline isn't usable at that point.
And remember that the CSI does not allow both output pads to be enabled
at the same time. If that were so then indeed there would be a problem,
because it would mean there is another active pipeline that requires the
CSI being powered on, but that's not the case.
I think this is consistent with the other entities as well, but I will
double check.
Steve
Russell King - ARM Linux
Mar 12, 2017, 4:00:05 PM3/12/17
to
On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote:
> If it's too difficult to get the imx219 csi-2 transmitter into the
> LP-11 state on power on, perhaps the csi-2 receiver can be a little
> more lenient on the transmitter and make the LP-11 timeout a warning
> instead of error-out.
>
> Can you try the attached change on top of the version 5 patchset?
>
> If that doesn't work then you're just going to have to fix the bug
> in imx219.
That patch gets me past that hurdle, only to reveal that there's another
> If it's too difficult to get the imx219 csi-2 transmitter into the
> LP-11 state on power on, perhaps the csi-2 receiver can be a little
> more lenient on the transmitter and make the LP-11 timeout a warning
> instead of error-out.
>
> Can you try the attached change on top of the version 5 patchset?
>
> If that doesn't work then you're just going to have to fix the bug
> in imx219.
issue:
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x00000200
imx219 0-0010: VT: line period 12385ns
imx219 0-0010: OP: pixclk 38500000Hz, 2 lanes, 308Mbps peak each
imx219 0-0010: OP: 3288 bits/line/lane act=10675ns lp/idle=1710ns
ipu1_csi0: csi_idmac_setup failed: -22
ipu1_csi0: pipeline start failed with -22
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1860 at /home/rmk/git/linux-rmk/drivers/media/v4l2-core/videobuf2-core.c:1340 vb2_start_streaming+0x124/0x1b4 [videobuf2_core]
Steve Longerbeam
Mar 12, 2017, 4:10:05 PM3/12/17
to
On 03/12/2017 12:47 PM, Russell King - ARM Linux wrote:
> Another issue.
>
> The "reboot and the /dev/video* devices come up in a completely
> different order" problem seems to exist with this version.
>
> The dot graph I supplied previously had "ipu1_csi0 capture" on
> /dev/video4. I've just rebooted, and now I find it's on
> /dev/video2 instead.
yet.
It could be as simple as passing a fixed device node # to
video_register_device(), but something tells me it won't be
that easy. But I'll get to this in next version.
Steve
Steve Longerbeam
Mar 12, 2017, 4:10:06 PM3/12/17
to
On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote:
> On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote:
>> If it's too difficult to get the imx219 csi-2 transmitter into the
>> LP-11 state on power on, perhaps the csi-2 receiver can be a little
>> more lenient on the transmitter and make the LP-11 timeout a warning
>> instead of error-out.
>>
>> Can you try the attached change on top of the version 5 patchset?
>>
>> If that doesn't work then you're just going to have to fix the bug
>> in imx219.
>
> That patch gets me past that hurdle, only to reveal that there's another
> issue:
bayer formats. Wait, didn't we fix this already? I've lost track.
Ah, right, we were going to move this support into the IPUv3 driver,
but in the meantime I think you had some patches to get around this.
Steve
Steve Longerbeam
Mar 12, 2017, 4:20:05 PM3/12/17
to
receiver.
It can enable all four of its output pads at once (if the input stream
contains all 4 virtual channels, the csi-2 receiver must support
demuxing all of them onto all 4 of its output pads).
But after more review, this should not be an issue. If a csi-2 sink
(a CSI or a CSI mux) link is disabled, the csi-2 receiver is no longer
reachable from that sink, so attempts to disable the csi-2 via that
path again is not possible. The other potential problem is disabling
from the csi-2's own sink pad, but in that case the csi-2 no longer
has a source, so again it makes sense to power off the csi-2 even
if it has enabled output pads.
Steve
Russell King - ARM Linux
Mar 12, 2017, 4:30:04 PM3/12/17
to
On Sun, Mar 12, 2017 at 01:05:06PM -0700, Steve Longerbeam wrote:
>
>
> On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote:
> >On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote:
> >>If it's too difficult to get the imx219 csi-2 transmitter into the
> >>LP-11 state on power on, perhaps the csi-2 receiver can be a little
> >>more lenient on the transmitter and make the LP-11 timeout a warning
> >>instead of error-out.
> >>
> >>Can you try the attached change on top of the version 5 patchset?
> >>
> >>If that doesn't work then you're just going to have to fix the bug
> >>in imx219.
> >
> >That patch gets me past that hurdle, only to reveal that there's another
> >issue:
>
> Yeah, ipu_cpmem_set_image() failed because it doesn't recognize the
> bayer formats. Wait, didn't we fix this already? I've lost track.
> Ah, right, we were going to move this support into the IPUv3 driver,
> but in the meantime I think you had some patches to get around this.
What I had was this patch for your v3. I never got to testing your
>
>
> On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote:
> >On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote:
> >>If it's too difficult to get the imx219 csi-2 transmitter into the
> >>LP-11 state on power on, perhaps the csi-2 receiver can be a little
> >>more lenient on the transmitter and make the LP-11 timeout a warning
> >>instead of error-out.
> >>
> >>Can you try the attached change on top of the version 5 patchset?
> >>
> >>If that doesn't work then you're just going to have to fix the bug
> >>in imx219.
> >
> >That patch gets me past that hurdle, only to reveal that there's another
> >issue:
>
> Yeah, ipu_cpmem_set_image() failed because it doesn't recognize the
> bayer formats. Wait, didn't we fix this already? I've lost track.
> Ah, right, we were going to move this support into the IPUv3 driver,
> but in the meantime I think you had some patches to get around this.
v4 because of the LP-11 problem.
In v5, you've changed to propagate the ipu_cpmem_set_image() error
code to avoid the resulting corruption, but that leaves the other bits
of this patch unaddressed, along my "media: imx: smfc: add support
for bayer formats" patch.
Your driver basically has no support for bayer formats.
diff --git a/drivers/staging/media/imx/imx-smfc.c b/drivers/staging/media/imx/imx-smfc.c
index 313732201a52..4351c0365cf4 100644
--- a/drivers/staging/media/imx/imx-smfc.c
+++ b/drivers/staging/media/imx/imx-smfc.c
@@ -234,11 +234,6 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv)
buf1 = imx_media_dma_buf_get_next_queued(priv->out_ring);
priv->next = buf1;
- image.phys0 = buf0->phys;
- image.phys1 = buf1->phys;
- ipu_cpmem_set_image(priv->smfc_ch, &image);
-
-
switch (image.pix.pixelformat) {
case V4L2_PIX_FMT_SBGGR8:
case V4L2_PIX_FMT_SGBRG8:
@@ -247,6 +242,10 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv)
burst_size = 8;
passthrough = true;
passthrough_bits = 8;
+ ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height);
+ ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline);
+ ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys);
+ ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys);
break;
case V4L2_PIX_FMT_SBGGR16:
@@ -256,9 +255,17 @@ static void imx_smfc_setup_channel(struct imx_smfc_priv *priv)
burst_size = 4;
passthrough = true;
passthrough_bits = 16;
+ ipu_cpmem_set_resolution(priv->smfc_ch, image.rect.width, image.rect.height);
+ ipu_cpmem_set_stride(priv->smfc_ch, image.pix.bytesperline);
+ ipu_cpmem_set_buffer(priv->smfc_ch, 0, buf0->phys);
+ ipu_cpmem_set_buffer(priv->smfc_ch, 1, buf1->phys);
break;
default:
+ image.phys0 = buf0->phys;
+ image.phys1 = buf1->phys;
+ ipu_cpmem_set_image(priv->smfc_ch, &image);
+
burst_size = (outfmt->width & 0xf) ? 8 : 16;
/*
Steve Longerbeam
Mar 12, 2017, 4:40:05 PM3/12/17
to
get reached in your case, multiple times? Yes I think there is a bug,
link_notify() is not checking if the link has already been disabled.
I will fix this. But I'm surprised media core's link_notify handling
doesn't do this.
Steve
Steve Longerbeam
Mar 12, 2017, 4:40:06 PM3/12/17
to
int __media_entity_setup_link(struct media_link *link, u32 flags)
{
...
if (link->flags == flags)
return 0;
...
}
What the heck. Anyway, I'll track this down.
Steve
>
> Steve
>
Russell King - ARM Linux
Mar 12, 2017, 4:50:05 PM3/12/17
to
On Sun, Mar 12, 2017 at 01:36:32PM -0700, Steve Longerbeam wrote:
> But hold on, if my logic is correct, then why did the CSI power-off
> get reached in your case, multiple times? Yes I think there is a bug,
> link_notify() is not checking if the link has already been disabled.
> I will fix this. But I'm surprised media core's link_notify handling
> doesn't do this.
Well, I think there's something incredibly fishy going on here. I
> But hold on, if my logic is correct, then why did the CSI power-off
> get reached in your case, multiple times? Yes I think there is a bug,
> link_notify() is not checking if the link has already been disabled.
> I will fix this. But I'm surprised media core's link_notify handling
> doesn't do this.
turned that dev_dbg() at the top of the function into a dev_info(),
and I get:
root@hbi2ex:~# dmesg |grep -A2 imx-ipuv3-csi
[ 53.370949] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
[ 53.371015] ------------[ cut here ]------------
[ 53.371075] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
--
[ 53.372624] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
[ 53.372637] ------------[ cut here ]------------
[ 53.372663] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
There isn't a power on event being generated before these two power
off events. I don't see a power on event even when I attempt to
start streaming either (which fails due to the lack of bayer
support.)
Mauro Carvalho Chehab
Mar 12, 2017, 5:00:04 PM3/12/17
to
Em Sun, 12 Mar 2017 19:47:00 +0000
Russell King - ARM Linux <li...@armlinux.org.uk> escreveu:
Didn't have time yet to read/comment the other e-mails in this thread.
Yet, as this is a simple issue, let me answer it first.
With regards to /dev/video?, the device number depends on the probing
order, with can be random on SoC drivers, due to the way OF works.
Yet, udev/systemd has some rules that provide an unique name for V4L
devices at /lib/udev/rules.d/60-persistent-v4l.rules. Basically, it
runs a small application (v4l_id) with creates a persistent symling
using rules like this:
KERNEL=="video*", ENV{ID_SERIAL}=="?*", SYMLINK+="v4l/by-id/$env{ID_BUS}-$env{ID_SERIAL}-video-index$attr{index}"
Those names are stored at /dev/v4l/by-path.
For example, on Exynos, we have:
$ ls -lctra /dev/v4l/by-path/
total 0
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-13e10000.video-scaler-video-index0 -> ../../video7
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-13e00000.video-scaler-video-index0 -> ../../video6
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f60000.jpeg-video-index0 -> ../../video4
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f50000.jpeg-video-index1 -> ../../video3
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f50000.jpeg-video-index0 -> ../../video2
drwxr-xr-x 3 root root 60 Mar 11 07:19 ..
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f60000.jpeg-video-index1 -> ../../video5
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11000000.codec-video-index1 -> ../../video1
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11000000.codec-video-index0 -> ../../video0
No matter what driver gets probed first, the above names should not
change.
So, if you want to write a script, the best is to use the /dev/v4l/by-path.
Unfortunately, gstreamer has some issues with that, as some of their plugins
don't seem to allow passing the name of the devnode, but just the number of
/dev/video?.
So, you need some script to convert from /dev/v4l/by-path/foo to
/dev/video?.
What I'm using on Exynos scripts is this logic:
NEEDED1=platform-13e00000.video-scaler-video-index0
DEV1=$(ls -l /dev/v4l/by-path/$NEEDED1|perl -ne ' print $1 if (m,/video(\d+),)')
Then, if I need to talk with this mem2mem driver using the v4l2video
convert plugin, I can launch gst with something like:
gst-launch-1.0 videotestsrc ! v4l2video${DEV1}convert ! fakesink
Thanks,
Mauro
Thanks,
Mauro
Russell King - ARM Linux <li...@armlinux.org.uk> escreveu:
Yet, as this is a simple issue, let me answer it first.
With regards to /dev/video?, the device number depends on the probing
order, with can be random on SoC drivers, due to the way OF works.
Yet, udev/systemd has some rules that provide an unique name for V4L
devices at /lib/udev/rules.d/60-persistent-v4l.rules. Basically, it
runs a small application (v4l_id) with creates a persistent symling
using rules like this:
KERNEL=="video*", ENV{ID_SERIAL}=="?*", SYMLINK+="v4l/by-id/$env{ID_BUS}-$env{ID_SERIAL}-video-index$attr{index}"
Those names are stored at /dev/v4l/by-path.
For example, on Exynos, we have:
$ ls -lctra /dev/v4l/by-path/
total 0
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-13e10000.video-scaler-video-index0 -> ../../video7
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-13e00000.video-scaler-video-index0 -> ../../video6
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f60000.jpeg-video-index0 -> ../../video4
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f50000.jpeg-video-index1 -> ../../video3
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f50000.jpeg-video-index0 -> ../../video2
drwxr-xr-x 3 root root 60 Mar 11 07:19 ..
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11f60000.jpeg-video-index1 -> ../../video5
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11000000.codec-video-index1 -> ../../video1
lrwxrwxrwx 1 root root 12 Mar 11 07:19 platform-11000000.codec-video-index0 -> ../../video0
No matter what driver gets probed first, the above names should not
change.
So, if you want to write a script, the best is to use the /dev/v4l/by-path.
Unfortunately, gstreamer has some issues with that, as some of their plugins
don't seem to allow passing the name of the devnode, but just the number of
/dev/video?.
So, you need some script to convert from /dev/v4l/by-path/foo to
/dev/video?.
What I'm using on Exynos scripts is this logic:
NEEDED1=platform-13e00000.video-scaler-video-index0
DEV1=$(ls -l /dev/v4l/by-path/$NEEDED1|perl -ne ' print $1 if (m,/video(\d+),)')
Then, if I need to talk with this mem2mem driver using the v4l2video
convert plugin, I can launch gst with something like:
gst-launch-1.0 videotestsrc ! v4l2video${DEV1}convert ! fakesink
Thanks,
Mauro
Thanks,
Mauro
Russell King - ARM Linux
Mar 12, 2017, 5:20:04 PM3/12/17
to
On Sun, Mar 12, 2017 at 05:59:28PM -0300, Mauro Carvalho Chehab wrote:
> Yet, udev/systemd has some rules that provide an unique name for V4L
> devices at /lib/udev/rules.d/60-persistent-v4l.rules. Basically, it
> runs a small application (v4l_id) with creates a persistent symling
> using rules like this:
>
> KERNEL=="video*", ENV{ID_SERIAL}=="?*", SYMLINK+="v4l/by-id/$env{ID_BUS}-$env{ID_SERIAL}-video-index$attr{index}"
>
> Those names are stored at /dev/v4l/by-path.
This doesn't help:
> Yet, udev/systemd has some rules that provide an unique name for V4L
> devices at /lib/udev/rules.d/60-persistent-v4l.rules. Basically, it
> runs a small application (v4l_id) with creates a persistent symling
> using rules like this:
>
> KERNEL=="video*", ENV{ID_SERIAL}=="?*", SYMLINK+="v4l/by-id/$env{ID_BUS}-$env{ID_SERIAL}-video-index$attr{index}"
>
> Those names are stored at /dev/v4l/by-path.
$ ls -Al /dev/v4l/by-id/
total 0
lrwxrwxrwx 1 root root 13 Mar 12 19:54 usb-Sonix_Technology_Co.__Ltd._USB_2.0_Camera-video-index0 -> ../../video10
$ ls -Al /dev/v4l/by-path/
total 0
lrwxrwxrwx 1 root root 12 Mar 12 19:54 platform-2040000.vpu-video-index0 -> ../../video0
lrwxrwxrwx 1 root root 12 Mar 12 19:54 platform-2040000.vpu-video-index1 -> ../../video1
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index0 -> ../../video2
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index1 -> ../../video3
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index2 -> ../../video4
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index3 -> ../../video5
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index4 -> ../../video6
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index5 -> ../../video7
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index6 -> ../../video8
lrwxrwxrwx 1 root root 12 Mar 12 20:53 platform-capture-subsystem-video-index7 -> ../../video9
lrwxrwxrwx 1 root root 13 Mar 12 19:54 platform-ci_hdrc.0-usb-0:1:1.0-video-index0 -> ../../video10
The problem is the "platform-capture-subsystem-video-index" entries.
These themselves change order. For instance, I now have:
- entity 72: ipu1_csi0 capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video6
which means it's platform-capture-subsystem-video-index4. Before, it
was platform-capture-subsystem-video-index2.
Russell King - ARM Linux
Mar 12, 2017, 5:20:04 PM3/12/17
to
On Sun, Mar 12, 2017 at 08:40:37PM +0000, Russell King - ARM Linux wrote:
> On Sun, Mar 12, 2017 at 01:36:32PM -0700, Steve Longerbeam wrote:
> > But hold on, if my logic is correct, then why did the CSI power-off
> > get reached in your case, multiple times? Yes I think there is a bug,
> > link_notify() is not checking if the link has already been disabled.
> > I will fix this. But I'm surprised media core's link_notify handling
> > doesn't do this.
>
> Well, I think there's something incredibly fishy going on here. I
> turned that dev_dbg() at the top of the function into a dev_info(),
> and I get:
>
> root@hbi2ex:~# dmesg |grep -A2 imx-ipuv3-csi
> [ 53.370949] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
> [ 53.371015] ------------[ cut here ]------------
> [ 53.371075] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
> --
> [ 53.372624] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
> [ 53.372637] ------------[ cut here ]------------
> [ 53.372663] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
>
> There isn't a power on event being generated before these two power
> off events. I don't see a power on event even when I attempt to
> start streaming either (which fails due to the lack of bayer
> support.)
Found it - my imx219 driver returns '1' from its s_power function when
> On Sun, Mar 12, 2017 at 01:36:32PM -0700, Steve Longerbeam wrote:
> > But hold on, if my logic is correct, then why did the CSI power-off
> > get reached in your case, multiple times? Yes I think there is a bug,
> > link_notify() is not checking if the link has already been disabled.
> > I will fix this. But I'm surprised media core's link_notify handling
> > doesn't do this.
>
> Well, I think there's something incredibly fishy going on here. I
> turned that dev_dbg() at the top of the function into a dev_info(),
> and I get:
>
> root@hbi2ex:~# dmesg |grep -A2 imx-ipuv3-csi
> [ 53.370949] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
> [ 53.371015] ------------[ cut here ]------------
> [ 53.371075] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
> --
> [ 53.372624] imx-ipuv3-csi imx-ipuv3-csi.0: power OFF
> [ 53.372637] ------------[ cut here ]------------
> [ 53.372663] WARNING: CPU: 0 PID: 1515 at drivers/staging/media/imx/imx-media-csi.c:806 csi_s_power+0xb8/0xd0 [imx_media_csi]
>
> There isn't a power on event being generated before these two power
> off events. I don't see a power on event even when I attempt to
> start streaming either (which fails due to the lack of bayer
> support.)
powering up, which triggers a bug in your code - when imx_media_set_power()
fails to power up, you call imx_media_set_power() telling it to power
everything off - including devices that are already powered off.
This is really bad news - s_power() may be called via other paths,
such as when the subdev is opened.
Mauro Carvalho Chehab
Mar 12, 2017, 6:20:05 PM3/12/17
to
Em Sun, 12 Mar 2017 21:13:24 +0000
That's a driver problem. v4l_id gets information to build the persistent
name from the result of VIDIOC_QUERYCAP.
In the case of Exynos gsc driver, for example, the information is here:
static int gsc_m2m_querycap(struct file *file, void *fh,
struct v4l2_capability *cap)
{
struct gsc_ctx *ctx = fh_to_ctx(fh);
struct gsc_dev *gsc = ctx->gsc_dev;
strlcpy(cap->driver, GSC_MODULE_NAME, sizeof(cap->driver));
strlcpy(cap->card, GSC_MODULE_NAME " gscaler", sizeof(cap->card));
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
dev_name(&gsc->pdev->dev));
cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE |
V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_VIDEO_OUTPUT_MPLANE;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
return 0;
}
See that the bus_info there is filled with:
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s", dev_name(&gsc->pdev->dev));
From the output you printed, it seems that the i.MX6 is just doing:
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:");
for some devices.
If you change the i.MX6 driver to do the same, you'll likely be able to
have unique names there too.
Regards,
Mauro
name from the result of VIDIOC_QUERYCAP.
In the case of Exynos gsc driver, for example, the information is here:
static int gsc_m2m_querycap(struct file *file, void *fh,
struct v4l2_capability *cap)
{
struct gsc_ctx *ctx = fh_to_ctx(fh);
struct gsc_dev *gsc = ctx->gsc_dev;
strlcpy(cap->driver, GSC_MODULE_NAME, sizeof(cap->driver));
strlcpy(cap->card, GSC_MODULE_NAME " gscaler", sizeof(cap->card));
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
dev_name(&gsc->pdev->dev));
cap->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE |
V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_VIDEO_OUTPUT_MPLANE;
cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
return 0;
}
See that the bus_info there is filled with:
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s", dev_name(&gsc->pdev->dev));
From the output you printed, it seems that the i.MX6 is just doing:
snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:");
for some devices.
If you change the i.MX6 driver to do the same, you'll likely be able to
have unique names there too.
Regards,
Mauro
Steve Longerbeam
Mar 13, 2017, 12:40:05 AM3/13/17
to
On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote:
> On Sun, Mar 12, 2017 at 01:05:06PM -0700, Steve Longerbeam wrote:
>>
>>
>> On 03/12/2017 12:57 PM, Russell King - ARM Linux wrote:
>>> On Sat, Mar 11, 2017 at 04:30:53PM -0800, Steve Longerbeam wrote:
>>>> If it's too difficult to get the imx219 csi-2 transmitter into the
>>>> LP-11 state on power on, perhaps the csi-2 receiver can be a little
>>>> more lenient on the transmitter and make the LP-11 timeout a warning
>>>> instead of error-out.
>>>>
>>>> Can you try the attached change on top of the version 5 patchset?
>>>>
>>>> If that doesn't work then you're just going to have to fix the bug
>>>> in imx219.
>>>
>>> That patch gets me past that hurdle, only to reveal that there's another
>>> issue:
>>
>> Yeah, ipu_cpmem_set_image() failed because it doesn't recognize the
>> bayer formats. Wait, didn't we fix this already? I've lost track.
>> Ah, right, we were going to move this support into the IPUv3 driver,
>> but in the meantime I think you had some patches to get around this.
>
> What I had was this patch for your v3. I never got to testing your
> v4 because of the LP-11 problem.
>
> In v5, you've changed to propagate the ipu_cpmem_set_image() error
> code to avoid the resulting corruption, but that leaves the other bits
> of this patch unaddressed, along my "media: imx: smfc: add support
> for bayer formats" patch.
>
> Your driver basically has no support for bayer formats.
I don't think there is anything more required of the driver at this
point to support bayer, the remaining work needs to happen in the IPUv3
driver.
I'll see if I have time to write that patch to IPUv3, but it's simple,
in fact what you wrote below can be translate directly into
ipu_cpmem_set_image(). There's a few other places bayer needs to be
treated in IPUv3, but it should be obvious by grepping for the
reference to pixel formats.
Steve
Russell King - ARM Linux
Mar 13, 2017, 4:20:05 AM3/13/17
to
On Sun, Mar 12, 2017 at 09:26:41PM -0700, Steve Longerbeam wrote:
> On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote:
> >What I had was this patch for your v3. I never got to testing your
> >v4 because of the LP-11 problem.
> >
> >In v5, you've changed to propagate the ipu_cpmem_set_image() error
> >code to avoid the resulting corruption, but that leaves the other bits
> >of this patch unaddressed, along my "media: imx: smfc: add support
> >for bayer formats" patch.
> >
> >Your driver basically has no support for bayer formats.
>
> You added the patches to this driver that adds the bayer support,
> I don't think there is anything more required of the driver at this
> point to support bayer, the remaining work needs to happen in the IPUv3
> driver.
There is more work, because the way you've merged my changes to
> On 03/12/2017 01:22 PM, Russell King - ARM Linux wrote:
> >What I had was this patch for your v3. I never got to testing your
> >v4 because of the LP-11 problem.
> >
> >In v5, you've changed to propagate the ipu_cpmem_set_image() error
> >code to avoid the resulting corruption, but that leaves the other bits
> >of this patch unaddressed, along my "media: imx: smfc: add support
> >for bayer formats" patch.
> >
> >Your driver basically has no support for bayer formats.
>
> You added the patches to this driver that adds the bayer support,
> I don't think there is anything more required of the driver at this
> point to support bayer, the remaining work needs to happen in the IPUv3
> driver.
imx_smfc_setup_channel() into csi_idmac_setup_channel() is wrong with
respect to the burst size.
You always set it to 8 or 16 depending on the width:
burst_size = (image.pix.width & 0xf) ? 8 : 16;
ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
and then you have my switch() statement which assigns burst_size.
My _tested_ code removed the above, added the switch, which had
a default case which reflected the above setting:
default:
burst_size = (outfmt->width & 0xf) ? 8 : 16;
and then went on to set the burst size _after_ the switch statement:
ipu_cpmem_set_burstsize(priv->smfc_ch, burst_size);
The effect is unchanged for non-bayer formats. For bayer formats, the
burst size is determined by the bayer data size.
So, even if it's appropriate to fix ipu_cpmem_set_image(), fixing the
above is still required.
I'm not convinced that fixing ipu_cpmem_set_image() is even the best
solution - it's not as trivial as it looks on the surface:
ipu_cpmem_set_resolution(ch, image->rect.width, image->rect.height);
ipu_cpmem_set_stride(ch, pix->bytesperline);
this is fine, it doesn't depend on the format. However, the next line:
ipu_cpmem_set_fmt(ch, v4l2_pix_fmt_to_drm_fourcc(pix->pixelformat));
does - v4l2_pix_fmt_to_drm_fourcc() is a locally defined function (it
isn't v4l2 code) that converts a v4l2 pixel format to a DRM fourcc.
DRM knows nothing about bayer formats, there aren't fourcc codes in
DRM for it. The result is that v4l2_pix_fmt_to_drm_fourcc() returns
-EINVAL cast to a u32, which gets passed unchecked into ipu_cpmem_set_fmt().
ipu_cpmem_set_fmt() won't recognise that, and also returns -EINVAL - and
it's a bug that this is not checked and propagated. If it is checked and
propagated, then we need this to support bayer formats, and I don't see
DRM people wanting bayer format fourcc codes added without there being
a real DRM driver wanting to use them.
Then there's the business of calculating the top-left offset of the image,
which for bayer always needs to be an even number of pixels - as this
function takes the top-left offset, it ought to respect it, but if it
doesn't meet this criteria, what should it do? csi_idmac_setup_channel()
always sets them to zero, but that's not really something that
ipu_cpmem_set_image() should assume.
Russell King - ARM Linux
Mar 13, 2017, 5:40:09 AM3/13/17
to
as I originally had. This seems to get me working capture, but might
break non-bayer passthrough mode:
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index fc0036aa84d0..df336971a009 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -314,14 +314,6 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
image.phys0 = phys[0];
image.phys1 = phys[1];
- ret = ipu_cpmem_set_image(priv->idmac_ch, &image);
- if (ret)
- return ret;
-
- burst_size = (image.pix.width & 0xf) ? 8 : 16;
-
- ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
-
/*
* Check for conditions that require the IPU to handle the
* data internally as generic data, aka passthrough mode:
@@ -346,15 +338,29 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
passthrough_bits = 16;
break;
default:
+ burst_size = (image.pix.width & 0xf) ? 8 : 16;
passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
sensor_ep->bus.parallel.bus_width >= 16);
passthrough_bits = 16;
break;
}
- if (passthrough)
+ if (passthrough) {
+ ipu_cpmem_set_resolution(priv->idmac_ch, image.rect.width,
+ image.rect.height);
+ ipu_cpmem_set_stride(priv->idmac_ch, image.pix.bytesperline);
+ ipu_cpmem_set_buffer(priv->idmac_ch, 0, image.phys0);
+ ipu_cpmem_set_buffer(priv->idmac_ch, 1, image.phys1);
+ ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
ipu_cpmem_set_format_passthrough(priv->idmac_ch,
passthrough_bits);
+ } else {
+ ret = ipu_cpmem_set_image(priv->idmac_ch, &image);
+ if (ret)
+ return ret;
+
+ ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
+ }
/*
* Set the channel for the direct CSI-->memory via SMFC
Hans Verkuil
Mar 13, 2017, 6:00:04 AM3/13/17
to
On 03/11/2017 07:20 PM, Steve Longerbeam wrote:
>>> Add an empty UAPI Kbuild file for media UAPI headers.
>>>
>>> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
>>
>> The existing V4L2 UAPI headers are under include/uapi/linux. Could you use
>> that directory instead?
>>
>> I actually wouldn't really object doing this but it should have been done in
>> 2002 or so when the first V4L2 header was added. Now the benefit is
>> questionable.
>>
>
> Agreed, I think the current headers should be moved to uapi/media
> eventually, but for now I'll go ahead and put under uapi/linux.
No, while in staging it shouldn't be exported.
Put it in include/linux and move it to uapi when this driver is mainlined.
I don't think we can move headers from uapi/linux to uapi/media, I'm sure it's
too late for that.
Regards,
Hans
>
>
> On 03/11/2017 05:49 AM, Sakari Ailus wrote:
>> Hi Steve,
>>
>> On Thu, Mar 09, 2017 at 08:53:01PM -0800, Steve Longerbeam wrote:
>
> On 03/11/2017 05:49 AM, Sakari Ailus wrote:
>> Hi Steve,
>>
>>> Add an empty UAPI Kbuild file for media UAPI headers.
>>>
>>> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
>>
>> The existing V4L2 UAPI headers are under include/uapi/linux. Could you use
>> that directory instead?
>>
>> I actually wouldn't really object doing this but it should have been done in
>> 2002 or so when the first V4L2 header was added. Now the benefit is
>> questionable.
>>
>
> Agreed, I think the current headers should be moved to uapi/media
> eventually, but for now I'll go ahead and put under uapi/linux.
No, while in staging it shouldn't be exported.
Put it in include/linux and move it to uapi when this driver is mainlined.
I don't think we can move headers from uapi/linux to uapi/media, I'm sure it's
too late for that.
Regards,
Hans
Hans Verkuil
Mar 13, 2017, 6:10:08 AM3/13/17
to
On 03/11/2017 07:14 PM, Steve Longerbeam wrote:
>
>
> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>> On 10/03/17 19:37, Steve Longerbeam wrote:
>
>
> On 03/11/2017 03:39 AM, Hans Verkuil wrote:
>>> Hi Hans,
>>>
>>> On 03/10/2017 04:03 AM, Hans Verkuil wrote:
>>>> On 10/03/17 05:52, Steve Longerbeam wrote:
>>>>> Add a new FRAME_INTERVAL_ERROR event to signal that a video capture or
>>>>> output device has measured an interval between the reception or transmit
>>>>> completion of two consecutive frames of video that is outside the nominal
>>>>> frame interval by some tolerance value.
>>>>
>>>> Reading back what was said on this I agree with Sakari that this doesn't
>>>> belong here.
>>>>
>>>> Userspace can detect this just as easily (if not easier) with a timeout.
>>>>
>>>
>>>
>>> Unfortunately measuring frame intervals from userland is not accurate
>>> enough for i.MX6.
>>>
>>> The issue here is that the IPUv3, specifically the CSI unit, can
>>> permanently lose vertical sync if there are truncated frames sent
>>> on the bt.656 bus. We have seen a single missing line of video cause
>>> loss of vertical sync. The only way to correct this is to shutdown
>>> the IPU capture hardware and restart, which can be accomplished
>>> simply by restarting streaming from userland.
>>>
>>> There are no other indicators from the sensor about these short
>>> frame events (believe me, we've exhausted all avenues with the ADV718x).
>>> And the IPUv3 DMA engine has no status indicators for short frames
>>> either. So the only way to detect them is by measuring frame intervals.
>>>
>>> The intervals have to be able to resolve a single line of missing video.
>>> With a PAL video source that requires better than 58 usec accuracy.
>>>
>>> There is too much uncertainty to resolve this at user level. The
>>> driver is able to resolve this by measuring intervals between hardware
>>> interrupts as long as interrupt latency is reasonably low, and we
>>> have another method using the i.MX6 hardware input capture support
>>> that can measure these intervals very accurately with no errors
>>> introduced by interrupt latency.
>>>
>>> I made this event a private event to imx-media driver in a previous
>>> iteration, so I can return it to a private event, but this can't be
>>> done at user level.
>>
>> It's fine to use an internal event as long as the end-user doesn't
>> see it. But if you lose vsyncs, then you never capture another frame,
>> right?
>
> No, that's not correct. By loss of vertical sync, I mean the IPU
> captures portions of two different frames, resulting in a permanent
> "split image", with one frame containing portions of two consecutive
> images. Or, the video rolls continuously, if you remember the old CRT
> television sets of yore, it's the same rolling effect.
>
>
>> So userspace can detect that (i.e. no new frames arrive) and
>> It's fine to use an internal event as long as the end-user doesn't
>> see it. But if you lose vsyncs, then you never capture another frame,
>> right?
>
> No, that's not correct. By loss of vertical sync, I mean the IPU
> captures portions of two different frames, resulting in a permanent
> "split image", with one frame containing portions of two consecutive
> images. Or, the video rolls continuously, if you remember the old CRT
> television sets of yore, it's the same rolling effect.
>
>
>> it can timeout on that. Or you detect it in the driver and restart there,
>> or call vb2_queue_error().
>>
>
> There is no timeout, the frames keep coming, but they are split images
> or rolling.
Ah, OK. That wasn't clear to me from the description.
>
>> Anything really as long as this event isn't user-visible :-)
>
> The event must be user visible, otherwise the user has no indication
> the error, and can't correct it by stream restart.
In that case the driver can detect this and call vb2_queue_error. It's
what it is there for.
The event doesn't help you since only this driver has this issue. So nobody
will watch this event, unless it is sw specifically written for this SoC.
Much better to call vb2_queue_error to signal a fatal error (which this
apparently is) since there are more drivers that do this, and vivid supports
triggering this condition as well.
Regards,
Hans
Russell King - ARM Linux
Mar 13, 2017, 6:50:05 AM3/13/17
to
an understanding of how this sensor works. Several of the registers
(such as the PLL setup [*]) require me to disable streaming on the
sensor while changing them.
This is something I've done many times while testing various ideas,
and is my primary way of figuring out and testing such things.
Whenever I resume streaming (provided I've let the sensor stop
streaming at a frame boundary) it resumes as if nothing happened. If I
stop the sensor mid-frame, then I get the rolling issue that Steve
reports, but once the top of the frame becomes aligned with the top of
the capture, everything then becomes stable again as if nothing happened.
The side effect of what you're proposing is that when I disable streaming
at the sensor by poking at its registers, rather than the capture just
stopping, an error is going to be delivered to gstreamer, and gstreamer
is going to exit, taking the entire capture process down.
This severely restricts the ability to be able to develop and test
sensor drivers.
So, I strongly disagree with you.
Loss of capture frames is not necessarily a fatal error - as I have been
saying repeatedly. In Steve's case, there's some unknown interaction
between the source and iMX6 hardware that is causing the instability,
but that is simply not true of other sources, and I oppose any idea that
we should cripple the iMX6 side of the capture based upon just one
hardware combination where this is a problem.
Steve suggested that the problem could be in the iMX6 CSI block - and I
note comparing Steve's code with the code in FSL's repository that there
are some changes that are missing in Steve's code to do with the CCIR656
sync code setup, particularly for >8 bit. The progressive CCIR656 8-bit
setup looks pretty similar though - but I think what needs to be asked
is whether the same problem is visible using the FSL/NXP vendor kernel.
* - the PLL setup is something that requires research at the moment.
Sony's official position (even to their customers) is that they do not
supply the necessary information, instead they expect customers to tell
them the capture settings they want, and Sony will throw the values into
a spreadsheet, and they'll supply the register settings back to the
customer. Hence, the only way to proceed with a generic driver for
this sensor is to experiment, and experimenting requires the ability to
pause the stream at the sensor while making changes. Take this away,
and we're stuck with the tables-of-register-settings-for-set-of-fixed-
capture-settings approach. I've made a lot of progress away from this
which is all down to the flexibility afforded by _not_ killing the
capture process.
Hans Verkuil
Mar 13, 2017, 7:00:05 AM3/13/17
to
call vb2_queue_error in this specific case. Sending an event that nobody
will know how to handle or what to do with is pretty pointless IMHO.
Let's just give him time to try and figure out the real issue here.
Regards,
Hans
Steve Longerbeam
Mar 13, 2017, 1:10:06 PM3/13/17
to
our customer to try and get to the bottom of it. I can go into a lot of
details from those trips, we probed the bt.656 bus with a logic analyzer
and I can share those results with anyone who asks. But the results of
those investigations indicate the CSI is not handling the SAV/EAV sync
codes correctly - if there is a shift in the line position at which
those codes occur, the CSI/IPU does not abort the frame capture DMA
and start from the new sync code position, it just continues to capture
lines until the programmed number of lines are transferred, hence you
get these split images. Freescale also informed us of a mechanism in the
IPU that will add lines if it detects these short frames, until the
programmed number of lines are reached. Apparently that is what creates
the rolling effect, but this rolling can last for up to a full minute,
which is completely unacceptable, it must be corrected as soon as
possible.
So the only thing we could come up with was to monitor frame intervals,
this is purely empirical, but we observed that frame intervals drop
by approx. one line time (~60 usec) when these short frames are
received. I don't really have any explanation for that but we take
advantage of that observation by sending this event to userspace so
the problem can be corrected immediately with a stream restart.
As I've said, the ADV718x does not provide _any_ indication via status
when the shift in the sync code position occurs. And the IPU is also
severely lacking in DMA completion status as well (no short packet
status like in USB for example). So the only way to detect this event
is by monitoring the frame intervals.
I will review differences in the CCIR code register setup from FSL's
repo that Russell pointed out, but I'm fairly sure those code registers
are setup correctly, there's not much room for variability in those
values. They only define the values of the sync codes, so the CSI can
detect them, those values are defined by the bt.656 spec.
Anyway, perhaps for now I can remove the event, but keep the FI
monitoring, and for now just report a kernel error message on
a detected bad FI.
Steve
Hans Verkuil
Mar 13, 2017, 1:20:05 PM3/13/17
to
is not present (or at least resolves itself very quickly) on the imx219.
If it is not possible, then a private event would be acceptable, but
needs to be carefully documented. After all, it is a workaround for a bug,
since otherwise there would be no need for this event.
Regards,
Hans
Steve Longerbeam
Mar 13, 2017, 5:50:06 PM3/13/17
to
not an issue for imx219. It is an issue for any sensor with a
parallel BT.656 interface.
>
> If it is not possible, then a private event would be acceptable, but
> needs to be carefully documented. After all, it is a workaround for a bug,
> since otherwise there would be no need for this event.
sure it is documented well. Yes it is a workaround, for a silicon bug,
although Freescale/NXP has not issued an errata for it yet AFAIK. THat
might be because they claim to handle it via this adding lines
mechanism, but that mechanism doesn't work well (long duration rolling),
or not at all (permanent split images).
Steve
Steve Longerbeam
Mar 13, 2017, 7:40:04 PM3/13/17
to
er, I meant I will integrate this patch. And verify/fix
possible breakage for non-bayer passthrough.
Steve
possible breakage for non-bayer passthrough.
Steve
Steve Longerbeam
Mar 13, 2017, 7:40:04 PM3/13/17
to
>
> I'm not convinced that fixing ipu_cpmem_set_image() is even the best
> solution - it's not as trivial as it looks on the surface:
>
> ipu_cpmem_set_resolution(ch, image->rect.width, image->rect.height);
> ipu_cpmem_set_stride(ch, pix->bytesperline);
>
> this is fine, it doesn't depend on the format. However, the next line:
>
> ipu_cpmem_set_fmt(ch, v4l2_pix_fmt_to_drm_fourcc(pix->pixelformat));
>
> does - v4l2_pix_fmt_to_drm_fourcc() is a locally defined function (it
> isn't v4l2 code) that converts a v4l2 pixel format to a DRM fourcc.
> DRM knows nothing about bayer formats, there aren't fourcc codes in
> DRM for it.
> The result is that v4l2_pix_fmt_to_drm_fourcc() returns
> -EINVAL cast to a u32, which gets passed unchecked into ipu_cpmem_set_fmt().
>
> ipu_cpmem_set_fmt() won't recognise that, and also returns -EINVAL - and
> it's a bug that this is not checked and propagated. If it is checked and
> propagated, then we need this to support bayer formats, and I don't see
> DRM people wanting bayer format fourcc codes added without there being
> a real DRM driver wanting to use them.
>
> Then there's the business of calculating the top-left offset of the image,
> which for bayer always needs to be an even number of pixels - as this
> function takes the top-left offset, it ought to respect it, but if it
> doesn't meet this criteria, what should it do? csi_idmac_setup_channel()
> always sets them to zero, but that's not really something that
> ipu_cpmem_set_image() should assume.
work for me.
We do need to address the issues you brought up in ipu_cpmem at
some point.
Steve
Nicolas Dufresne
Mar 14, 2017, 12:30:05 PM3/14/17
to
for an higher level programs to handle this. Nothing prevents from
adding something of course, but the errors are really un-specific, so
it would be something pretty blind. For what it has been tested, this
case was never met, usually the error is triggered by a USB camera
being un-plugged, a driver failure or even a firmware crash. Most of
the time, this is not recoverable.
My main concern here based on what I'm reading, is that this driver is
not even able to notice immediately that a produced frame was corrupted
(because it's out of sync). From usability perspective, this is really
bad. Can't the driver derive a clock from some irq and calculate for
each frame if the timing was correct ? And if not mark the buffer with
V4L2_BUF_FLAG_ERROR ?
>
> This severely restricts the ability to be able to develop and test
> sensor drivers.
>
> So, I strongly disagree with you.
>
> Loss of capture frames is not necessarily a fatal error - as I have been
> saying repeatedly. In Steve's case, there's some unknown interaction
> between the source and iMX6 hardware that is causing the instability,
> but that is simply not true of other sources, and I oppose any idea that
> we should cripple the iMX6 side of the capture based upon just one
> hardware combination where this is a problem.
Though, the driver is well aware, and mark the buffers with
V4L2_BUF_FLAG_ERROR.
Steve Longerbeam
Mar 14, 2017, 12:50:04 PM3/14/17
to
occurs when disrupting the analog video source signal in some
way (by unplugging the RCA cable from the ADV718x connector
for example).
Second, there is no DMA status support in i.MX6 to catch these
shifted bt.656 codes, and the ADV718x does not provide any
status indicators of this event either. So monitoring frame intervals
is the only solution available, until FSL/NXP issues a new silicon rev.
> Can't the driver derive a clock from some irq and calculate for
> each frame if the timing was correct ?
> And if not mark the buffer with
> V4L2_BUF_FLAG_ERROR ?
unspecific.
Steve
Russell King - ARM Linux
Mar 14, 2017, 12:50:10 PM3/14/17
to
On Tue, Mar 14, 2017 at 12:21:31PM -0400, Nicolas Dufresne wrote:
> My main concern here based on what I'm reading, is that this driver is
> not even able to notice immediately that a produced frame was corrupted
> (because it's out of sync). From usability perspective, this is really
> bad. Can't the driver derive a clock from some irq and calculate for
> each frame if the timing was correct ? And if not mark the buffer with
> V4L2_BUF_FLAG_ERROR ?
One of the issues of measuring timing with IRQs is the fact that the
> My main concern here based on what I'm reading, is that this driver is
> not even able to notice immediately that a produced frame was corrupted
> (because it's out of sync). From usability perspective, this is really
> bad. Can't the driver derive a clock from some irq and calculate for
> each frame if the timing was correct ? And if not mark the buffer with
> V4L2_BUF_FLAG_ERROR ?
IRQ subsystem only allows one IRQ to run at a time. If an IRQ takes
a relatively long time to process, then it throws the timing of other
IRQs out.
If you're going to decide that a buffer should be marked in error on
the basis of an interrupt arriving late, this can trigger spuriously.
It wasn't that long ago that USB HID was regularly eating something
like 20ms of interrupt time... that's been solved, but that doesn't
mean all cases are solved - there are still interrupt handlers in the
kernel that are on the order of milliseconds to complete.
Given the quality I observe of some USB serial devices (eg, running at
115200 baud, but feeling like they deliver characters to userspace at
9600 baud) I wouldn't be surprised if some USB serial drivers eat a lot
of IRQ time... and if so, all it'll take is to plug such a device in
to disrupt capture.
That sounds way too fragile to me.
Steve Longerbeam
Mar 14, 2017, 1:00:14 PM3/14/17
to
On 03/14/2017 09:47 AM, Russell King - ARM Linux wrote:
> On Tue, Mar 14, 2017 at 12:21:31PM -0400, Nicolas Dufresne wrote:
>> My main concern here based on what I'm reading, is that this driver is
>> not even able to notice immediately that a produced frame was corrupted
>> (because it's out of sync). From usability perspective, this is really
>> bad. Can't the driver derive a clock from some irq and calculate for
>> each frame if the timing was correct ? And if not mark the buffer with
>> V4L2_BUF_FLAG_ERROR ?
> One of the issues of measuring timing with IRQs is the fact that the
> IRQ subsystem only allows one IRQ to run at a time. If an IRQ takes
> a relatively long time to process, then it throws the timing of other
> IRQs out.
>
> If you're going to decide that a buffer should be marked in error on
> the basis of an interrupt arriving late, this can trigger spuriously.
>
> It wasn't that long ago that USB HID was regularly eating something
> like 20ms of interrupt time... that's been solved, but that doesn't
> mean all cases are solved - there are still interrupt handlers in the
> kernel that are on the order of milliseconds to complete.
>
> Given the quality I observe of some USB serial devices (eg, running at
> 115200 baud, but feeling like they deliver characters to userspace at
> 9600 baud) I wouldn't be surprised if some USB serial drivers eat a lot
> of IRQ time... and if so, all it'll take is to plug such a device in
> to disrupt capture.
>
> That sounds way too fragile to me.
Steve
Steve Longerbeam
Mar 14, 2017, 1:10:09 PM3/14/17
to
snprintf(cap->bus_info, sizeof(cap->bus_info),
"platform:%s", dev_name(priv->dev));
dev_name(priv->dev) is the device name of the attached subdev.
So the bus_info string, at least for attached CSI subdevs, should
be "platform:imx-ipuv3-csi".
Maybe there is something else missing, I haven't had a chance to
look at this yet.
Steve
Steve Longerbeam
Mar 14, 2017, 1:30:06 PM3/14/17
to
On error, it needs to backout by calling s_power(off) as it is doing,
but not
through the whole pipeline, but needs to stop at the subdev encountered
just before the subdev that failed. This was causing the s_power()
imbalance.
I will fix.
>
> This is really bad news - s_power() may be called via other paths,
> such as when the subdev is opened.
as it should, and the caller from the other paths has not created an
imbalance.
Steve
Pavel Machek
Mar 14, 2017, 2:30:06 PM3/14/17
to
best for driver debugging.
And yes, I guess you can have #ifdef or module parameter or something
switching for behaviour you prefer when you are debugging. But for
production, vb2_queue_error() seems to be the right solution.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Steve Longerbeam
Mar 15, 2017, 2:50:05 PM3/15/17
to
On 03/10/2017 11:17 AM, Fabio Estevam wrote:
> On Fri, Mar 10, 2017 at 3:59 PM, Troy Kisky
> <troy....@boundarydevices.com> wrote:
>> On 3/9/2017 8:52 PM, Steve Longerbeam wrote:
>>> There is a pin conflict with GPIO_6. This pin functions as a power
>>> input pin to the OV5642 camera sensor, but ENET uses it as the h/w
>>> workaround for erratum ERR006687, to wake-up the ARM cores on normal
>>> RX and TX packet done events. So we need to remove the h/w workaround
>>> to support the OV5642. The result is that the CPUidle driver will no
>>> longer allow entering the deep idle states on the sabrelite.
>>>
>>> This is a partial revert of
>>>
>>> commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
>>> interrupt.")
>>> commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
>>> for ERR006687")
>>>
>>> Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
>>> ---
>>> arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ----
>>> 1 file changed, 4 deletions(-)
>>>
>>> diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
>>> index 8413179..89dce27 100644
>>> --- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
>>> +++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
>>> @@ -270,9 +270,6 @@
>>> txd1-skew-ps = <0>;
>>> txd2-skew-ps = <0>;
>>> txd3-skew-ps = <0>;
>>
>> How about
>>
>> +#if !IS_ENABLED(CONFIG_VIDEO_OV5642)
>
> Or maybe just create a new device tree for using the camera, like
> imx6q-sabrelite-camera.dts.
>
> This way we can keep the FEC erratum for the existing sabrelite dtb's.
Is it really necessary to keep the erratum in sabrelite dts? Because the
sabrelite is a _reference_ platform, vendors use this dts as a working
example of how to configure their imx6-based hardware. So as a working
example, it should contain as much example hardware config as possible
as a guide. If a vendor does not require OV5642 support and requires
the lower power consumption that the erratum workaround provides, they
can refer to other example imx6 dts files which still implement the
erratum, or look at the git log of this file.
Steve
Rob Herring
Mar 16, 2017, 5:30:05 PM3/16/17
to
On Thu, Mar 09, 2017 at 08:52:41PM -0800, Steve Longerbeam wrote:
> From: Philipp Zabel <p.z...@pengutronix.de>
>
> Add bindings documentation for the video multiplexer device.
>
> Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
> Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
> .../bindings/media/video-multiplexer.txt | 59 ++++++++++++++++++++++
> 1 file changed, 59 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
Acked-by: Rob Herring <ro...@kernel.org>
> From: Philipp Zabel <p.z...@pengutronix.de>
>
> Add bindings documentation for the video multiplexer device.
>
> Signed-off-by: Sascha Hauer <s.h...@pengutronix.de>
> Signed-off-by: Philipp Zabel <p.z...@pengutronix.de>
> .../bindings/media/video-multiplexer.txt | 59 ++++++++++++++++++++++
> 1 file changed, 59 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/video-multiplexer.txt
Acked-by: Rob Herring <ro...@kernel.org>
Sakari Ailus
Mar 16, 2017, 6:30:04 PM3/16/17
to
On Sat, Mar 11, 2017 at 12:31:24PM -0800, Steve Longerbeam wrote:
> >>If the pads on both sides of a link specify a frame interval, then
> >>those frame intervals should match. Create the exported function
> >>v4l2_subdev_link_validate_frame_interval() to verify this. This
> >>function can be called in a subdevice's media_entity_operations
> >>or v4l2_subdev_pad_ops link_validate callbacks.
> >>
> >>Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
> >
> >If your only goal is to configure frame dropping on a sub-device, I suggest
> >to implement s_frame_interval() on the pads of that sub-device only. The
> >frames are then dropped according to the configured frame rates between the
> >sink and source pads. Say, configuring sink for 1/30 s and source 1/15 would
> >drop half of the incoming frames.
> >
> >Considering that supporting specific frame interval on most sub-devices adds
> >no value or is not the interface through which it the frame rate configured,
> >I think it is overkill to change the link validation to expect otherwise.
>
>
> Well, while I think this function might still have validity in the future, I
> do agree with you that a subdev that has no control over
> frame rate has no business implementing the get|set ops.
>
> In the imx-media subdevs, the only one that can affect frame rate (via
> frame skipping) is the CSI. So I'll go ahead and remove the
> [gs]_frame_interval ops from the others. I can remove this patch as
> a result.
Agreed.
--
Sakari Ailus
e-mail: sakari...@iki.fi XMPP: sai...@retiisi.org.uk
>
>
> On 03/11/2017 05:41 AM, Sakari Ailus wrote:
> >Hi Steve,
> >
> >On Thu, Mar 09, 2017 at 08:52:58PM -0800, Steve Longerbeam wrote:
>
> On 03/11/2017 05:41 AM, Sakari Ailus wrote:
> >Hi Steve,
> >
> >>If the pads on both sides of a link specify a frame interval, then
> >>those frame intervals should match. Create the exported function
> >>v4l2_subdev_link_validate_frame_interval() to verify this. This
> >>function can be called in a subdevice's media_entity_operations
> >>or v4l2_subdev_pad_ops link_validate callbacks.
> >>
> >>Signed-off-by: Steve Longerbeam <steve_lo...@mentor.com>
> >
> >If your only goal is to configure frame dropping on a sub-device, I suggest
> >to implement s_frame_interval() on the pads of that sub-device only. The
> >frames are then dropped according to the configured frame rates between the
> >sink and source pads. Say, configuring sink for 1/30 s and source 1/15 would
> >drop half of the incoming frames.
> >
> >Considering that supporting specific frame interval on most sub-devices adds
> >no value or is not the interface through which it the frame rate configured,
> >I think it is overkill to change the link validation to expect otherwise.
>
>
> Well, while I think this function might still have validity in the future, I
> do agree with you that a subdev that has no control over
> frame rate has no business implementing the get|set ops.
>
> In the imx-media subdevs, the only one that can affect frame rate (via
> frame skipping) is the CSI. So I'll go ahead and remove the
> [gs]_frame_interval ops from the others. I can remove this patch as
> a result.
Agreed.
--
Sakari Ailus
e-mail: sakari...@iki.fi XMPP: sai...@retiisi.org.uk
It is loading more messages.
0 new messages