[PATCH v7 0/9] Qualcomm video decoder/encoder driver

[Stanimir Varbanov]

Hi all,

Here is seventh version of the patch-set - no functional changes in
v4l2 APIs.

The changes since v6 are.
* changes in DT binding document - moved memory-region DT property
in video-codec node - see 2/9.
* improved recovery mechanism.
* fixed various issues found during testing.

Build dependencies:
- qcom_scm_set_remote_state is merged in Linux 4.11-rc1
- qcom mdt_loader is merged in Linux 4.11-rc1

regards,
Stan

Stanimir Varbanov (9):
media: v4l2-mem2mem: extend m2m APIs for more accurate buffer
management
doc: DT: venus: binding document for Qualcomm video driver
MAINTAINERS: Add Qualcomm Venus video accelerator driver
media: venus: adding core part and helper functions
media: venus: vdec: add video decoder files
media: venus: venc: add video encoder files
media: venus: hfi: add Host Firmware Interface (HFI)
media: venus: hfi: add Venus HFI files
media: venus: enable building of Venus video driver

.../devicetree/bindings/media/qcom,venus.txt | 107 ++
MAINTAINERS | 8 +
drivers/media/platform/Kconfig | 14 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/qcom/venus/Makefile | 11 +
drivers/media/platform/qcom/venus/core.c | 386 +++++
drivers/media/platform/qcom/venus/core.h | 306 ++++
drivers/media/platform/qcom/venus/firmware.c | 107 ++
drivers/media/platform/qcom/venus/firmware.h | 22 +
drivers/media/platform/qcom/venus/helpers.c | 632 ++++++++
drivers/media/platform/qcom/venus/helpers.h | 41 +
drivers/media/platform/qcom/venus/hfi.c | 520 +++++++
drivers/media/platform/qcom/venus/hfi.h | 174 +++
drivers/media/platform/qcom/venus/hfi_cmds.c | 1256 ++++++++++++++++
drivers/media/platform/qcom/venus/hfi_cmds.h | 304 ++++
drivers/media/platform/qcom/venus/hfi_helper.h | 1050 +++++++++++++
drivers/media/platform/qcom/venus/hfi_msgs.c | 1058 +++++++++++++
drivers/media/platform/qcom/venus/hfi_msgs.h | 283 ++++
drivers/media/platform/qcom/venus/hfi_venus.c | 1570 ++++++++++++++++++++
drivers/media/platform/qcom/venus/hfi_venus.h | 23 +
drivers/media/platform/qcom/venus/hfi_venus_io.h | 113 ++
drivers/media/platform/qcom/venus/vdec.c | 1091 ++++++++++++++
drivers/media/platform/qcom/venus/vdec.h | 23 +
drivers/media/platform/qcom/venus/vdec_ctrls.c | 149 ++
drivers/media/platform/qcom/venus/venc.c | 1231 +++++++++++++++
drivers/media/platform/qcom/venus/venc.h | 23 +
drivers/media/platform/qcom/venus/venc_ctrls.c | 258 ++++
drivers/media/v4l2-core/v4l2-mem2mem.c | 37 +
include/media/v4l2-mem2mem.h | 92 ++
29 files changed, 10891 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/qcom,venus.txt
create mode 100644 drivers/media/platform/qcom/venus/Makefile
create mode 100644 drivers/media/platform/qcom/venus/core.c
create mode 100644 drivers/media/platform/qcom/venus/core.h
create mode 100644 drivers/media/platform/qcom/venus/firmware.c
create mode 100644 drivers/media/platform/qcom/venus/firmware.h
create mode 100644 drivers/media/platform/qcom/venus/helpers.c
create mode 100644 drivers/media/platform/qcom/venus/helpers.h
create mode 100644 drivers/media/platform/qcom/venus/hfi.c
create mode 100644 drivers/media/platform/qcom/venus/hfi.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_cmds.c
create mode 100644 drivers/media/platform/qcom/venus/hfi_cmds.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_helper.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_msgs.c
create mode 100644 drivers/media/platform/qcom/venus/hfi_msgs.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_venus.c
create mode 100644 drivers/media/platform/qcom/venus/hfi_venus.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_venus_io.h
create mode 100644 drivers/media/platform/qcom/venus/vdec.c
create mode 100644 drivers/media/platform/qcom/venus/vdec.h
create mode 100644 drivers/media/platform/qcom/venus/vdec_ctrls.c
create mode 100644 drivers/media/platform/qcom/venus/venc.c
create mode 100644 drivers/media/platform/qcom/venus/venc.h
create mode 100644 drivers/media/platform/qcom/venus/venc_ctrls.c

--
2.7.4

[Stanimir Varbanov]

Add an entry for Venus video encoder/decoder accelerator driver.

Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
---
MAINTAINERS | 8 ++++++++
1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 83a42ef1d1a7..cce2537d4d00 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10400,6 +10400,14 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/rkuo/linux-hexagon-kernel.g
S: Supported
F: arch/hexagon/

+QUALCOMM VENUS VIDEO ACCELERATOR DRIVER
+M: Stanimir Varbanov <stanimir...@linaro.org>
+L: linux...@vger.kernel.org
+L: linux-...@vger.kernel.org
+T: git git://linuxtv.org/media_tree.git
+S: Maintained
+F: drivers/media/platform/qcom/venus/
+
QUALCOMM WCN36XX WIRELESS DRIVER
M: Eugene Krasnikov <k.eug...@gmail.com>
L: wcn...@lists.infradead.org
--
2.7.4

[Stanimir Varbanov]

This consists of video decoder implementation plus decoder
controls.

Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
---

drivers/media/platform/qcom/venus/vdec.c | 1091 ++++++++++++++++++++++++
drivers/media/platform/qcom/venus/vdec.h | 23 +
drivers/media/platform/qcom/venus/vdec_ctrls.c | 149 ++++
3 files changed, 1263 insertions(+)

create mode 100644 drivers/media/platform/qcom/venus/vdec.c
create mode 100644 drivers/media/platform/qcom/venus/vdec.h
create mode 100644 drivers/media/platform/qcom/venus/vdec_ctrls.c

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
new file mode 100644
index 000000000000..ec5203f2ba81
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -0,0 +1,1091 @@
+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-sg.h>
+
+#include "hfi_venus_io.h"
+#include "core.h"
+#include "helpers.h"
+#include "vdec.h"
+
+static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
+{
+ u32 y_stride, uv_stride, y_plane;
+ u32 y_sclines, uv_sclines, uv_plane;
+ u32 size;
+
+ y_stride = ALIGN(width, 128);
+ uv_stride = ALIGN(width, 128);
+ y_sclines = ALIGN(height, 32);
+ uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+ y_plane = y_stride * y_sclines;
+ uv_plane = uv_stride * uv_sclines + SZ_4K;
+ size = y_plane + uv_plane + SZ_8K;
+
+ return ALIGN(size, SZ_4K);
+}
+
+static u32 get_framesize_compressed(unsigned int width, unsigned int height)
+{
+ return ((width * height * 3 / 2) / 2) + 128;
+}
+
+static const struct venus_format vdec_formats[] = {
+ {
+ .pixfmt = V4L2_PIX_FMT_NV12,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_MPEG4,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_MPEG2,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_H263,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_G,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_VC1_ANNEX_L,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_VP8,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_XVID,
+ .num_planes = 1,
+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ },
+};
+
+static const struct venus_format *find_format(u32 pixfmt, u32 type)
+{
+ const struct venus_format *fmt = vdec_formats;
+ unsigned int size = ARRAY_SIZE(vdec_formats);
+ unsigned int i;
+
+ for (i = 0; i < size; i++) {
+ if (fmt[i].pixfmt == pixfmt)
+ break;
+ }
+
+ if (i == size || fmt[i].type != type)
+ return NULL;
+
+ return &fmt[i];
+}
+
+static const struct venus_format *find_format_by_index(int index, u32 type)
+{
+ const struct venus_format *fmt = vdec_formats;
+ unsigned int size = ARRAY_SIZE(vdec_formats);
+ int i, k = 0;
+
+ if (index < 0 || index > size)
+ return NULL;
+
+ for (i = 0; i < size; i++) {
+ if (fmt[i].type != type)
+ continue;
+ if (k == index)
+ break;
+ k++;
+ }
+
+ if (i == size)
+ return NULL;
+
+ return &fmt[i];
+}
+
+static const struct venus_format *
+vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
+ const struct venus_format *fmt;
+ unsigned int p;
+
+ memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+ fmt = find_format(pixmp->pixelformat, f->type);
+ if (!fmt) {
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ pixmp->pixelformat = V4L2_PIX_FMT_NV12;
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ pixmp->pixelformat = V4L2_PIX_FMT_H264;
+ else
+ return NULL;
+ fmt = find_format(pixmp->pixelformat, f->type);
+ pixmp->width = 1280;
+ pixmp->height = 720;
+ }
+
+ pixmp->width = clamp(pixmp->width, inst->cap_width.min,
+ inst->cap_width.max);
+ pixmp->height = clamp(pixmp->height, inst->cap_height.min,
+ inst->cap_height.max);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ pixmp->height = ALIGN(pixmp->height, 32);
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+ pixmp->num_planes = fmt->num_planes;
+ pixmp->flags = 0;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ for (p = 0; p < pixmp->num_planes; p++) {
+ pfmt[p].sizeimage =
+ get_framesize_uncompressed(p, pixmp->width,
+ pixmp->height);
+ pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
+ }
+ } else {
+ pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
+ pixmp->height);
+ pfmt[0].bytesperline = 0;
+ }
+
+ return fmt;
+}
+
+static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct venus_inst *inst = to_inst(file);
+
+ vdec_try_fmt_common(inst, f);
+
+ return 0;
+}
+
+static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct venus_inst *inst = to_inst(file);
+ const struct venus_format *fmt = NULL;
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ fmt = inst->fmt_cap;
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ fmt = inst->fmt_out;
+
+ if (inst->reconfig) {
+ struct v4l2_format format = {};
+
+ inst->out_width = inst->reconfig_width;
+ inst->out_height = inst->reconfig_height;
+ inst->reconfig = false;
+
+ format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ format.fmt.pix_mp.pixelformat = inst->fmt_cap->pixfmt;
+ format.fmt.pix_mp.width = inst->out_width;
+ format.fmt.pix_mp.height = inst->out_height;
+
+ vdec_try_fmt_common(inst, &format);
+
+ inst->width = format.fmt.pix_mp.width;
+ inst->height = format.fmt.pix_mp.height;
+ }
+
+ pixmp->pixelformat = fmt->pixfmt;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ pixmp->width = inst->width;
+ pixmp->height = inst->height;
+ pixmp->colorspace = inst->colorspace;
+ pixmp->ycbcr_enc = inst->ycbcr_enc;
+ pixmp->quantization = inst->quantization;
+ pixmp->xfer_func = inst->xfer_func;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ pixmp->width = inst->out_width;
+ pixmp->height = inst->out_height;
+ }
+
+ vdec_try_fmt_common(inst, f);
+
+ return 0;
+}
+
+static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct venus_inst *inst = to_inst(file);
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_pix_format_mplane orig_pixmp;
+ const struct venus_format *fmt;
+ struct v4l2_format format;
+ u32 pixfmt_out = 0, pixfmt_cap = 0;
+
+ orig_pixmp = *pixmp;
+
+ fmt = vdec_try_fmt_common(inst, f);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ pixfmt_out = pixmp->pixelformat;
+ pixfmt_cap = inst->fmt_cap->pixfmt;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ pixfmt_cap = pixmp->pixelformat;
+ pixfmt_out = inst->fmt_out->pixfmt;
+ }
+
+ memset(&format, 0, sizeof(format));
+
+ format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ format.fmt.pix_mp.pixelformat = pixfmt_out;
+ format.fmt.pix_mp.width = orig_pixmp.width;
+ format.fmt.pix_mp.height = orig_pixmp.height;
+ vdec_try_fmt_common(inst, &format);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ inst->out_width = format.fmt.pix_mp.width;
+ inst->out_height = format.fmt.pix_mp.height;
+ inst->colorspace = pixmp->colorspace;
+ inst->ycbcr_enc = pixmp->ycbcr_enc;
+ inst->quantization = pixmp->quantization;
+ inst->xfer_func = pixmp->xfer_func;
+ }
+
+ memset(&format, 0, sizeof(format));
+
+ format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ format.fmt.pix_mp.pixelformat = pixfmt_cap;
+ format.fmt.pix_mp.width = orig_pixmp.width;
+ format.fmt.pix_mp.height = orig_pixmp.height;
+ vdec_try_fmt_common(inst, &format);
+
+ inst->width = format.fmt.pix_mp.width;
+ inst->height = format.fmt.pix_mp.height;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ inst->fmt_out = fmt;
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ inst->fmt_cap = fmt;
+
+ return 0;
+}
+
+static int
+vdec_g_selection(struct file *file, void *fh, struct v4l2_selection *s)
+{
+ struct venus_inst *inst = to_inst(file);
+
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return -EINVAL;
+ s->r.width = inst->out_width;
+ s->r.height = inst->out_height;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+ s->r.width = inst->width;
+ s->r.height = inst->height;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+ s->r.width = inst->out_width;
+ s->r.height = inst->out_height;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ s->r.top = 0;
+ s->r.left = 0;
+
+ return 0;
+}
+
+static int
+vdec_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+ strlcpy(cap->driver, "qcom-venus", sizeof(cap->driver));
+ strlcpy(cap->card, "Qualcomm Venus video decoder", sizeof(cap->card));
+ strlcpy(cap->bus_info, "platform:qcom-venus", sizeof(cap->bus_info));
+
+ return 0;
+}
+
+static int vdec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ const struct venus_format *fmt;
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ fmt = find_format_by_index(f->index, f->type);
+ if (!fmt)
+ return -EINVAL;
+
+ f->pixelformat = fmt->pixfmt;
+
+ return 0;
+}
+
+static int vdec_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+ struct venus_inst *inst = to_inst(file);
+ struct v4l2_captureparm *cap = &a->parm.capture;
+ struct v4l2_fract *timeperframe = &cap->timeperframe;
+ u64 us_per_frame, fps;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ return -EINVAL;
+
+ memset(cap->reserved, 0, sizeof(cap->reserved));
+ if (!timeperframe->denominator)
+ timeperframe->denominator = inst->timeperframe.denominator;
+ if (!timeperframe->numerator)
+ timeperframe->numerator = inst->timeperframe.numerator;
+ cap->readbuffers = 0;
+ cap->extendedmode = 0;
+ cap->capability = V4L2_CAP_TIMEPERFRAME;
+ us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+ do_div(us_per_frame, timeperframe->denominator);
+
+ if (!us_per_frame)
+ return -EINVAL;
+
+ fps = (u64)USEC_PER_SEC;
+ do_div(fps, us_per_frame);
+
+ inst->fps = fps;
+ inst->timeperframe = *timeperframe;
+
+ return 0;
+}
+
+static int vdec_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct venus_inst *inst = to_inst(file);
+ const struct venus_format *fmt;
+
+ fmt = find_format(fsize->pixel_format,
+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ if (!fmt) {
+ fmt = find_format(fsize->pixel_format,
+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ if (!fmt)
+ return -EINVAL;
+ }
+
+ if (fsize->index)
+ return -EINVAL;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+
+ fsize->stepwise.min_width = inst->cap_width.min;
+ fsize->stepwise.max_width = inst->cap_width.max;
+ fsize->stepwise.step_width = inst->cap_width.step_size;
+ fsize->stepwise.min_height = inst->cap_height.min;
+ fsize->stepwise.max_height = inst->cap_height.max;
+ fsize->stepwise.step_height = inst->cap_height.step_size;
+
+ return 0;
+}
+
+static int vdec_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ return v4l2_event_subscribe(fh, sub, 2, NULL);
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_src_change_event_subscribe(fh, sub);
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subscribe_event(fh, sub);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
+ .vidioc_querycap = vdec_querycap,
+ .vidioc_enum_fmt_vid_cap_mplane = vdec_enum_fmt,
+ .vidioc_enum_fmt_vid_out_mplane = vdec_enum_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = vdec_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vdec_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = vdec_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vdec_g_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = vdec_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = vdec_try_fmt,
+ .vidioc_g_selection = vdec_g_selection,
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+ .vidioc_s_parm = vdec_s_parm,
+ .vidioc_enum_framesizes = vdec_enum_framesizes,
+ .vidioc_subscribe_event = vdec_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int vdec_set_properties(struct venus_inst *inst)
+{
+ struct vdec_controls *ctr = &inst->controls.dec;
+ struct venus_core *core = inst->core;
+ struct hfi_enable en = { .enable = 1 };
+ u32 ptype;
+ int ret;
+
+ ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
+ ret = hfi_session_set_property(inst, ptype, &en);
+ if (ret)
+ return ret;
+
+ if (core->res->hfi_version == HFI_VERSION_3XX) {
+ struct hfi_buffer_alloc_mode mode;
+
+ ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
+ mode.type = HFI_BUFFER_OUTPUT;
+ mode.mode = HFI_BUFFER_MODE_DYNAMIC;
+
+ ret = hfi_session_set_property(inst, ptype, &mode);
+ if (ret)
+ return ret;
+ }
+
+ if (ctr->post_loop_deb_mode) {
+ ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
+ en.enable = 1;
+ ret = hfi_session_set_property(inst, ptype, &en);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int vdec_init_session(struct venus_inst *inst)
+{
+ int ret;
+
+ ret = hfi_session_init(inst, inst->fmt_out->pixfmt);
+ if (ret)
+ return ret;
+
+ ret = helper_set_input_resolution(inst, inst->out_width,
+ inst->out_height);
+ if (ret)
+ goto deinit;
+
+ ret = helper_set_color_format(inst, inst->fmt_cap->pixfmt);
+ if (ret)
+ goto deinit;
+
+ return 0;
+deinit:
+ hfi_session_deinit(inst);
+ return ret;
+}
+
+static int vdec_cap_num_buffers(struct venus_inst *inst, unsigned int *num)
+{
+ struct hfi_buffer_requirements bufreq;
+ int ret;
+
+ ret = vdec_init_session(inst);
+ if (ret)
+ return ret;
+
+ ret = helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+
+ *num = bufreq.count_actual;
+
+ hfi_session_deinit(inst);
+
+ return ret;
+}
+
+static int vdec_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[])
+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);
+ unsigned int p, num;
+ int ret = 0;
+
+ if (*num_planes) {
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ *num_planes != inst->fmt_out->num_planes)
+ return -EINVAL;
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ *num_planes != inst->fmt_cap->num_planes)
+ return -EINVAL;
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ sizes[0] < inst->input_buf_size)
+ return -EINVAL;
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ sizes[0] < inst->output_buf_size)
+ return -EINVAL;
+
+ if (inst->core->res->hfi_version == HFI_VERSION_LEGACY)
+ return -EINVAL;
+
+ return 0;
+ }
+
+ switch (q->type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ *num_planes = inst->fmt_out->num_planes;
+ sizes[0] = get_framesize_compressed(inst->out_width,
+ inst->out_height);
+ inst->input_buf_size = sizes[0];
+ inst->num_input_bufs = *num_buffers;
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ *num_planes = inst->fmt_cap->num_planes;
+
+ ret = vdec_cap_num_buffers(inst, &num);
+ if (ret)
+ break;
+
+ *num_buffers = max(*num_buffers, num);
+
+ for (p = 0; p < *num_planes; p++)
+ sizes[p] = get_framesize_uncompressed(p, inst->width,
+ inst->height);
+
+ inst->num_output_bufs = *num_buffers;
+ inst->output_buf_size = sizes[0];
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static int vdec_verify_conf(struct venus_inst *inst)
+{
+ struct hfi_buffer_requirements bufreq;
+ int ret;
+
+ if (!inst->num_input_bufs || !inst->num_output_bufs)
+ return -EINVAL;
+
+ ret = helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+ if (ret)
+ return ret;
+
+ if (inst->num_output_bufs < bufreq.count_actual ||
+ inst->num_output_bufs < bufreq.count_min)
+ return -EINVAL;
+
+ ret = helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
+ if (ret)
+ return ret;
+
+ if (inst->num_input_bufs < bufreq.count_min)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);
+ struct venus_core *core = inst->core;
+ u32 ptype;
+ int ret;
+
+ mutex_lock(&inst->lock);
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ inst->streamon_out = 1;
+ else
+ inst->streamon_cap = 1;
+
+ if (!(inst->streamon_out & inst->streamon_cap)) {
+ mutex_unlock(&inst->lock);
+ return 0;
+ }
+
+ inst->reconfig = false;
+ inst->sequence = 0;
+ inst->codec_cfg = false;
+
+ ret = vdec_init_session(inst);
+ if (ret)
+ goto bufs_done;
+
+ ret = vdec_set_properties(inst);
+ if (ret)
+ goto deinit_sess;
+
+ if (core->res->hfi_version == HFI_VERSION_3XX) {
+ struct hfi_buffer_size_actual buf_sz;
+
+ ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
+ buf_sz.type = HFI_BUFFER_OUTPUT;
+ buf_sz.size = inst->output_buf_size;
+
+ ret = hfi_session_set_property(inst, ptype, &buf_sz);
+ if (ret)
+ goto deinit_sess;
+ }
+
+ ret = vdec_verify_conf(inst);
+ if (ret)
+ goto deinit_sess;
+
+ ret = helper_set_num_bufs(inst, inst->num_input_bufs,
+ inst->num_output_bufs);
+ if (ret)
+ goto deinit_sess;
+
+ ret = helper_vb2_start_streaming(inst);
+ if (ret)
+ goto deinit_sess;
+
+ mutex_unlock(&inst->lock);
+
+ return 0;
+
+deinit_sess:
+ hfi_session_deinit(inst);
+bufs_done:
+ helper_buffers_done(inst, VB2_BUF_STATE_QUEUED);
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ inst->streamon_out = 0;
+ else
+ inst->streamon_cap = 0;
+ mutex_unlock(&inst->lock);
+ return ret;
+}
+
+static const struct vb2_ops vdec_vb2_ops = {
+ .queue_setup = vdec_queue_setup,
+ .buf_init = helper_vb2_buf_init,
+ .buf_prepare = helper_vb2_buf_prepare,
+ .start_streaming = vdec_start_streaming,
+ .stop_streaming = helper_vb2_stop_streaming,
+ .buf_queue = helper_vb2_buf_queue,
+};
+
+static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
+ u32 tag, u32 bytesused, u32 data_offset, u32 flags,
+ u64 timestamp_us)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct vb2_buffer *vb;
+ unsigned int type;
+
+ if (buf_type == HFI_BUFFER_INPUT)
+ type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ else
+ type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+
+ vbuf = helper_find_buf(inst, type, tag);
+ if (!vbuf)
+ return;
+
+ vbuf->flags = flags;
+
+ if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ vb = &vbuf->vb2_buf;
+ vb->planes[0].bytesused =
+ max_t(unsigned int, inst->output_buf_size, bytesused);
+ vb->planes[0].data_offset = data_offset;
+ vb->timestamp = timestamp_us * NSEC_PER_USEC;
+ vbuf->sequence = inst->sequence++;
+
+ if (vbuf->flags & V4L2_BUF_FLAG_LAST) {
+ const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
+
+ v4l2_event_queue_fh(&inst->fh, &ev);
+ }
+ }
+
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
+}
+
+static void vdec_event_notify(struct venus_inst *inst, u32 event,
+ struct hfi_event_data *data)
+{
+ struct venus_core *core = inst->core;
+ struct device *dev = core->dev_dec;
+ static const struct v4l2_event ev = {
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION };
+
+ switch (event) {
+ case EVT_SESSION_ERROR:
+ inst->session_error = true;
+ dev_err(dev, "dec: event session error %x\n", inst->error);
+ break;
+ case EVT_SYS_EVENT_CHANGE:
+ switch (data->event_type) {
+ case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
+ hfi_session_continue(inst);
+ dev_dbg(dev, "event sufficient resources\n");
+ break;
+ case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
+ inst->reconfig_height = data->height;
+ inst->reconfig_width = data->width;
+ inst->reconfig = true;
+
+ v4l2_event_queue_fh(&inst->fh, &ev);
+
+ dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
+ data->width, data->height);
+ break;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+static const struct hfi_inst_ops vdec_hfi_ops = {
+ .buf_done = vdec_buf_done,
+ .event_notify = vdec_event_notify,
+};
+
+static void vdec_inst_init(struct venus_inst *inst)
+{
+ inst->fmt_out = &vdec_formats[6];
+ inst->fmt_cap = &vdec_formats[0];
+ inst->width = 1280;
+ inst->height = ALIGN(720, 32);
+ inst->out_width = 1280;
+ inst->out_height = 720;
+ inst->fps = 30;
+ inst->timeperframe.numerator = 1;
+ inst->timeperframe.denominator = 30;
+
+ inst->cap_width.min = 64;
+ if (inst->core->res->hfi_version == HFI_VERSION_LEGACY)
+ inst->cap_width.max = 1920;
+ else
+ inst->cap_width.max = 3840;
+ inst->cap_width.step_size = 1;
+ inst->cap_height.min = 64;
+ if (inst->core->res->hfi_version == HFI_VERSION_LEGACY)
+ inst->cap_height.max = ALIGN(1080, 32);
+ else
+ inst->cap_height.max = ALIGN(2160, 32);
+ inst->cap_height.step_size = 1;
+ inst->cap_framerate.min = 1;
+ inst->cap_framerate.max = 30;
+ inst->cap_framerate.step_size = 1;
+ inst->cap_mbs_per_frame.min = 16;
+ inst->cap_mbs_per_frame.max = 8160;
+}
+
+static const struct v4l2_m2m_ops vdec_m2m_ops = {
+ .device_run = helper_m2m_device_run,
+ .job_abort = helper_m2m_job_abort,
+};
+
+static int m2m_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct venus_inst *inst = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->ops = &vdec_vb2_ops;
+ src_vq->mem_ops = &vb2_dma_sg_memops;
+ src_vq->drv_priv = inst;
+ src_vq->buf_struct_size = sizeof(struct venus_buffer);
+ src_vq->allow_zero_bytesused = 1;
+ src_vq->min_buffers_needed = 1;
+ src_vq->dev = inst->core->dev;
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->ops = &vdec_vb2_ops;
+ dst_vq->mem_ops = &vb2_dma_sg_memops;
+ dst_vq->drv_priv = inst;
+ dst_vq->buf_struct_size = sizeof(struct venus_buffer);
+ dst_vq->allow_zero_bytesused = 1;
+ dst_vq->min_buffers_needed = 1;
+ dst_vq->dev = inst->core->dev;
+ ret = vb2_queue_init(dst_vq);
+ if (ret) {
+ vb2_queue_release(src_vq);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int vdec_open(struct file *file)
+{
+ struct venus_core *core = video_drvdata(file);
+ struct venus_inst *inst;
+ int ret;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&inst->registeredbufs);
+ INIT_LIST_HEAD(&inst->internalbufs);
+ INIT_LIST_HEAD(&inst->list);
+ mutex_init(&inst->lock);
+
+ inst->core = core;
+ inst->session_type = VIDC_SESSION_TYPE_DEC;
+
+ ret = pm_runtime_get_sync(core->dev_dec);
+ if (ret < 0)
+ goto err_free_inst;
+
+ ret = vdec_ctrl_init(inst);
+ if (ret)
+ goto err_put_sync;
+
+ ret = hfi_session_create(inst, &vdec_hfi_ops);
+ if (ret)
+ goto err_ctrl_deinit;
+
+ vdec_inst_init(inst);
+
+ /*
+ * create m2m device for every instance, the m2m context scheduling
+ * is made by firmware side so we do not need to care about.
+ */
+ inst->m2m_dev = v4l2_m2m_init(&vdec_m2m_ops);
+ if (IS_ERR(inst->m2m_dev)) {
+ ret = PTR_ERR(inst->m2m_dev);
+ goto err_session_destroy;
+ }
+
+ inst->m2m_ctx = v4l2_m2m_ctx_init(inst->m2m_dev, inst, m2m_queue_init);
+ if (IS_ERR(inst->m2m_ctx)) {
+ ret = PTR_ERR(inst->m2m_ctx);
+ goto err_m2m_release;
+ }
+
+ v4l2_fh_init(&inst->fh, core->vdev_dec);
+
+ inst->fh.ctrl_handler = &inst->ctrl_handler;
+ v4l2_fh_add(&inst->fh);
+ inst->fh.m2m_ctx = inst->m2m_ctx;
+ file->private_data = &inst->fh;
+
+ return 0;
+
+err_m2m_release:
+ v4l2_m2m_release(inst->m2m_dev);
+err_session_destroy:
+ hfi_session_destroy(inst);
+err_ctrl_deinit:
+ vdec_ctrl_deinit(inst);
+err_put_sync:
+ pm_runtime_put_sync(core->dev_dec);
+err_free_inst:
+ kfree(inst);
+ return ret;
+}
+
+static int vdec_close(struct file *file)
+{
+ struct venus_inst *inst = to_inst(file);
+
+ v4l2_m2m_ctx_release(inst->m2m_ctx);
+ v4l2_m2m_release(inst->m2m_dev);
+ vdec_ctrl_deinit(inst);
+ hfi_session_destroy(inst);
+ mutex_destroy(&inst->lock);
+ v4l2_fh_del(&inst->fh);
+ v4l2_fh_exit(&inst->fh);
+ kfree(inst);
+
+ pm_runtime_put_sync(inst->core->dev_dec);
+ return 0;
+}
+
+static const struct v4l2_file_operations vdec_fops = {
+ .owner = THIS_MODULE,
+ .open = vdec_open,
+ .release = vdec_close,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = v4l2_m2m_fop_poll,
+ .mmap = v4l2_m2m_fop_mmap,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static int vdec_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct video_device *vdev;
+ struct venus_core *core;
+ int ret;
+
+ if (!dev->parent)
+ return -EPROBE_DEFER;
+
+ core = dev_get_drvdata(dev->parent);
+ if (!core)
+ return -EPROBE_DEFER;
+
+ if (core->res->hfi_version == HFI_VERSION_3XX) {
+ core->core0_clk = devm_clk_get(dev, "core");
+ if (IS_ERR(core->core0_clk))
+ return PTR_ERR(core->core0_clk);
+ }
+
+ platform_set_drvdata(pdev, core);
+
+ vdev = video_device_alloc();
+ if (!vdev)
+ return -ENOMEM;
+
+ vdev->release = video_device_release;
+ vdev->fops = &vdec_fops;
+ vdev->ioctl_ops = &vdec_ioctl_ops;
+ vdev->vfl_dir = VFL_DIR_M2M;
+ vdev->v4l2_dev = &core->v4l2_dev;
+ vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+
+ ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_dec = vdev;
+ core->dev_dec = dev;
+
+ video_set_drvdata(vdev, core);
+ pm_runtime_enable(dev);
+
+ return 0;
+
+err_vdev_release:
+ video_device_release(vdev);
+ return ret;
+}
+
+static int vdec_remove(struct platform_device *pdev)
+{
+ struct venus_core *core = dev_get_drvdata(pdev->dev.parent);
+
+ video_unregister_device(core->vdev_dec);
+ pm_runtime_disable(core->dev_dec);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int vdec_runtime_suspend(struct device *dev)
+{
+ struct venus_core *core = dev_get_drvdata(dev);
+
+ if (core->res->hfi_version == HFI_VERSION_LEGACY)
+ return 0;
+
+ writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+ clk_disable_unprepare(core->core0_clk);
+ writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+ return 0;
+}
+
+static int vdec_runtime_resume(struct device *dev)
+{
+ struct venus_core *core = dev_get_drvdata(dev);
+ int ret;
+
+ if (core->res->hfi_version == HFI_VERSION_LEGACY)
+ return 0;
+
+ writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+ ret = clk_prepare_enable(core->core0_clk);
+ writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+ return ret;
+}
+#endif
+
+static const struct dev_pm_ops vdec_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(vdec_runtime_suspend, vdec_runtime_resume, NULL)
+};
+
+static const struct of_device_id vdec_dt_match[] = {
+ { .compatible = "venus-decoder" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, vdec_dt_match);
+
+static struct platform_driver qcom_venus_dec_driver = {
+ .probe = vdec_probe,
+ .remove = vdec_remove,
+ .driver = {
+ .name = "qcom-venus-decoder",
+ .of_match_table = vdec_dt_match,
+ .pm = &vdec_pm_ops,
+ },
+};
+module_platform_driver(qcom_venus_dec_driver);
+
+MODULE_ALIAS("platform:qcom-venus-decoder");
+MODULE_DESCRIPTION("Qualcomm Venus video decoder driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/venus/vdec.h
new file mode 100644
index 000000000000..84b672c54d02
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/vdec.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+#ifndef __VENUS_VDEC_H__
+#define __VENUS_VDEC_H__
+
+struct venus_inst;
+
+int vdec_ctrl_init(struct venus_inst *inst);
+void vdec_ctrl_deinit(struct venus_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/venus/vdec_ctrls.c
new file mode 100644
index 000000000000..c91e2ec37ea4
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/vdec_ctrls.c
@@ -0,0 +1,149 @@
+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+
+#include "core.h"
+
+static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct venus_inst *inst = ctrl_to_inst(ctrl);
+ struct vdec_controls *ctr = &inst->controls.dec;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
+ ctr->post_loop_deb_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+ ctr->profile = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+ ctr->level = ctrl->val;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct venus_inst *inst = ctrl_to_inst(ctrl);
+ struct vdec_controls *ctr = &inst->controls.dec;
+ union hfi_get_property hprop;
+ u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
+ int ret;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+ ret = hfi_session_get_property(inst, ptype, &hprop);
+ if (!ret)
+ ctr->profile = hprop.profile_level.profile;
+ ctrl->val = ctr->profile;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+ ret = hfi_session_get_property(inst, ptype, &hprop);
+ if (!ret)
+ ctr->level = hprop.profile_level.level;
+ ctrl->val = ctr->level;
+ break;
+ case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER:
+ ctrl->val = ctr->post_loop_deb_mode;
+ break;
+ default:
+ return -EINVAL;
+ };
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops vdec_ctrl_ops = {
+ .s_ctrl = vdec_op_s_ctrl,
+ .g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+};
+
+int vdec_ctrl_init(struct venus_inst *inst)
+{
+ struct v4l2_ctrl *ctrl;
+ int ret;
+
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 6);
+ if (ret)
+ return ret;
+
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+ V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
+ ~((1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
+ (1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)),
+ V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+ V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
+ 0, V4L2_MPEG_VIDEO_MPEG4_LEVEL_0);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
+ ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)),
+ V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
+ 0, V4L2_MPEG_VIDEO_H264_LEVEL_1_0);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_VPX_PROFILE, 0, 3, 1, 0);
+ if (ctrl)
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &vdec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER, 0, 1, 1, 0);
+
+ ret = inst->ctrl_handler.error;
+ if (ret) {
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
+ return ret;
+ }
+
+ return 0;
+}
+
+void vdec_ctrl_deinit(struct venus_inst *inst)
+{
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
+}
--
2.7.4

[Stanimir Varbanov]

This adds encoder part of the driver plus encoder controls.

Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
---

drivers/media/platform/qcom/venus/venc.c | 1231 ++++++++++++++++++++++++
drivers/media/platform/qcom/venus/venc.h | 23 +
drivers/media/platform/qcom/venus/venc_ctrls.c | 258 +++++
3 files changed, 1512 insertions(+)

create mode 100644 drivers/media/platform/qcom/venus/venc.c
create mode 100644 drivers/media/platform/qcom/venus/venc.h
create mode 100644 drivers/media/platform/qcom/venus/venc_ctrls.c

diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
new file mode 100644
index 000000000000..dddfc3554953
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -0,0 +1,1231 @@

+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>

+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-sg.h>

+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>

+
+#include "hfi_venus_io.h"
+#include "core.h"
+#include "helpers.h"

+#include "venc.h"
+
+#define NUM_B_FRAMES_MAX 4

+
+static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
+{
+ u32 y_stride, uv_stride, y_plane;
+ u32 y_sclines, uv_sclines, uv_plane;
+ u32 size;
+
+ y_stride = ALIGN(width, 128);
+ uv_stride = ALIGN(width, 128);
+ y_sclines = ALIGN(height, 32);
+ uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+ y_plane = y_stride * y_sclines;
+ uv_plane = uv_stride * uv_sclines + SZ_4K;
+ size = y_plane + uv_plane + SZ_8K;

+ size = ALIGN(size, SZ_4K);
+
+ return size;
+}
+
+static u32 get_framesize_compressed(u32 width, u32 height)
+{
+ u32 sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2 / 2;
+
+ return ALIGN(sz, SZ_4K);
+}
+
+static const struct venus_format venc_formats[] = {

+ {
+ .pixfmt = V4L2_PIX_FMT_NV12,
+ .num_planes = 1,

+ .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ }, {
+ .pixfmt = V4L2_PIX_FMT_MPEG4,
+ .num_planes = 1,

+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ }, {

+ .pixfmt = V4L2_PIX_FMT_H263,
+ .num_planes = 1,

+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ }, {

+ .pixfmt = V4L2_PIX_FMT_H264,
+ .num_planes = 1,

+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ }, {

+ .pixfmt = V4L2_PIX_FMT_VP8,
+ .num_planes = 1,

+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ },
+};
+
+static const struct venus_format *find_format(u32 pixfmt, int type)
+{
+ const struct venus_format *fmt = venc_formats;
+ unsigned int size = ARRAY_SIZE(venc_formats);

+ unsigned int i;
+
+ for (i = 0; i < size; i++) {
+ if (fmt[i].pixfmt == pixfmt)
+ break;
+ }
+
+ if (i == size || fmt[i].type != type)
+ return NULL;
+
+ return &fmt[i];
+}
+

+static const struct venus_format *find_format_by_index(int index, int type)
+{
+ const struct venus_format *fmt = venc_formats;
+ unsigned int size = ARRAY_SIZE(venc_formats);

+ int i, k = 0;
+
+ if (index < 0 || index > size)
+ return NULL;
+
+ for (i = 0; i < size; i++) {
+ if (fmt[i].type != type)
+ continue;
+ if (k == index)
+ break;
+ k++;
+ }
+
+ if (i == size)
+ return NULL;
+
+ return &fmt[i];
+}
+

+static int venc_v4l2_to_hfi(int id, int value)
+{
+ switch (id) {
+ case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+ switch (value) {
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_0:
+ default:
+ return HFI_MPEG4_LEVEL_0;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_0B:
+ return HFI_MPEG4_LEVEL_0b;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_1:
+ return HFI_MPEG4_LEVEL_1;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_2:
+ return HFI_MPEG4_LEVEL_2;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_3:
+ return HFI_MPEG4_LEVEL_3;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_4:
+ return HFI_MPEG4_LEVEL_4;
+ case V4L2_MPEG_VIDEO_MPEG4_LEVEL_5:
+ return HFI_MPEG4_LEVEL_5;
+ }
+ case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+ switch (value) {
+ case V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE:
+ default:
+ return HFI_MPEG4_PROFILE_SIMPLE;
+ case V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE:
+ return HFI_MPEG4_PROFILE_ADVANCEDSIMPLE;
+ }
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ switch (value) {
+ case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE:
+ return HFI_H264_PROFILE_BASELINE;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE:
+ return HFI_H264_PROFILE_CONSTRAINED_BASE;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN:
+ return HFI_H264_PROFILE_MAIN;
+ case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH:
+ default:
+ return HFI_H264_PROFILE_HIGH;
+ }
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ switch (value) {
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
+ return HFI_H264_LEVEL_1;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1B:
+ return HFI_H264_LEVEL_1b;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_1:
+ return HFI_H264_LEVEL_11;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_2:
+ return HFI_H264_LEVEL_12;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_1_3:
+ return HFI_H264_LEVEL_13;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_0:
+ return HFI_H264_LEVEL_2;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_1:
+ return HFI_H264_LEVEL_21;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_2_2:
+ return HFI_H264_LEVEL_22;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_0:
+ return HFI_H264_LEVEL_3;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_1:
+ return HFI_H264_LEVEL_31;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_3_2:
+ return HFI_H264_LEVEL_32;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_0:
+ return HFI_H264_LEVEL_4;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_1:
+ return HFI_H264_LEVEL_41;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_4_2:
+ return HFI_H264_LEVEL_42;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_5_0:
+ default:
+ return HFI_H264_LEVEL_5;
+ case V4L2_MPEG_VIDEO_H264_LEVEL_5_1:
+ return HFI_H264_LEVEL_51;
+ }
+ case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
+ switch (value) {
+ case V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC:
+ default:
+ return HFI_H264_ENTROPY_CAVLC;
+ case V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC:
+ return HFI_H264_ENTROPY_CABAC;
+ }
+ case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+ switch (value) {
+ case 0:
+ default:
+ return HFI_VPX_PROFILE_VERSION_0;
+ case 1:
+ return HFI_VPX_PROFILE_VERSION_1;
+ case 2:
+ return HFI_VPX_PROFILE_VERSION_2;
+ case 3:
+ return HFI_VPX_PROFILE_VERSION_3;
+ }

+ }
+
+ return 0;
+}
+
+static int

+venc_querycap(struct file *file, void *fh, struct v4l2_capability *cap)

+{
+ strlcpy(cap->driver, "qcom-venus", sizeof(cap->driver));

+ strlcpy(cap->card, "Qualcomm Venus video encoder", sizeof(cap->card));

+ strlcpy(cap->bus_info, "platform:qcom-venus", sizeof(cap->bus_info));
+
+ return 0;
+}
+

+static int venc_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)

+{
+ const struct venus_format *fmt;
+

+ fmt = find_format_by_index(f->index, f->type);
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ if (!fmt)
+ return -EINVAL;
+
+ f->pixelformat = fmt->pixfmt;
+
+ return 0;
+}
+

+static const struct venus_format *
+venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)

+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
+ const struct venus_format *fmt;
+ unsigned int p;
+
+ memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+ fmt = find_format(pixmp->pixelformat, f->type);
+ if (!fmt) {
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)

+ pixmp->pixelformat = V4L2_PIX_FMT_H264;

+ else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)

+ pixmp->pixelformat = V4L2_PIX_FMT_NV12;

+ else
+ return NULL;
+ fmt = find_format(pixmp->pixelformat, f->type);
+ pixmp->width = 1280;
+ pixmp->height = 720;
+ }
+
+ pixmp->width = clamp(pixmp->width, inst->cap_width.min,
+ inst->cap_width.max);
+ pixmp->height = clamp(pixmp->height, inst->cap_height.min,
+ inst->cap_height.max);
+

+ pixmp->height = ALIGN(pixmp->height, 32);
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+ pixmp->num_planes = fmt->num_planes;
+ pixmp->flags = 0;
+

+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {

+ for (p = 0; p < pixmp->num_planes; p++) {
+ pfmt[p].sizeimage =
+ get_framesize_uncompressed(p, pixmp->width,
+ pixmp->height);
+
+ pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
+ }
+ } else {
+ pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
+ pixmp->height);
+ pfmt[0].bytesperline = 0;
+ }
+
+ return fmt;
+}
+

+static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f)

+{
+ struct venus_inst *inst = to_inst(file);
+

+ venc_try_fmt_common(inst, f);

+
+ return 0;
+}
+

+static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)

+{
+ struct venus_inst *inst = to_inst(file);
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_pix_format_mplane orig_pixmp;
+ const struct venus_format *fmt;
+ struct v4l2_format format;
+ u32 pixfmt_out = 0, pixfmt_cap = 0;
+
+ orig_pixmp = *pixmp;
+

+ fmt = venc_try_fmt_common(inst, f);

+ if (!fmt)
+ return -EINVAL;
+

+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ pixfmt_out = pixmp->pixelformat;
+ pixfmt_cap = inst->fmt_cap->pixfmt;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ pixfmt_cap = pixmp->pixelformat;
+ pixfmt_out = inst->fmt_out->pixfmt;
+ }
+
+ memset(&format, 0, sizeof(format));
+
+ format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ format.fmt.pix_mp.pixelformat = pixfmt_out;
+ format.fmt.pix_mp.width = orig_pixmp.width;
+ format.fmt.pix_mp.height = orig_pixmp.height;

+ venc_try_fmt_common(inst, &format);

+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ inst->out_width = format.fmt.pix_mp.width;
+ inst->out_height = format.fmt.pix_mp.height;
+ inst->colorspace = pixmp->colorspace;
+ inst->ycbcr_enc = pixmp->ycbcr_enc;
+ inst->quantization = pixmp->quantization;
+ inst->xfer_func = pixmp->xfer_func;
+ }
+
+ memset(&format, 0, sizeof(format));
+
+ format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ format.fmt.pix_mp.pixelformat = pixfmt_cap;
+ format.fmt.pix_mp.width = orig_pixmp.width;
+ format.fmt.pix_mp.height = orig_pixmp.height;

+ venc_try_fmt_common(inst, &format);

+
+ inst->width = format.fmt.pix_mp.width;
+ inst->height = format.fmt.pix_mp.height;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ inst->fmt_out = fmt;
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ inst->fmt_cap = fmt;
+
+ return 0;
+}
+

+static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{

+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;

+ struct venus_inst *inst = to_inst(file);
+ const struct venus_format *fmt;
+

+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ fmt = inst->fmt_cap;
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ fmt = inst->fmt_out;

+ else
+ return -EINVAL;
+

+ pixmp->pixelformat = fmt->pixfmt;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ pixmp->width = inst->width;
+ pixmp->height = inst->height;
+ pixmp->colorspace = inst->colorspace;
+ pixmp->ycbcr_enc = inst->ycbcr_enc;
+ pixmp->quantization = inst->quantization;
+ pixmp->xfer_func = inst->xfer_func;
+ } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ pixmp->width = inst->out_width;
+ pixmp->height = inst->out_height;
+ }
+

+ venc_try_fmt_common(inst, f);

+
+ return 0;
+}
+
+static int

+venc_g_selection(struct file *file, void *fh, struct v4l2_selection *s)

+{
+ struct venus_inst *inst = to_inst(file);
+

+ if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

+ return -EINVAL;
+
+ switch (s->target) {

+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP_BOUNDS:

+ s->r.width = inst->width;
+ s->r.height = inst->height;
+ break;

+ case V4L2_SEL_TGT_CROP:

+ s->r.width = inst->out_width;
+ s->r.height = inst->out_height;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ s->r.top = 0;
+ s->r.left = 0;
+
+ return 0;
+}
+
+static int

+venc_s_selection(struct file *file, void *fh, struct v4l2_selection *s)

+{
+ struct venus_inst *inst = to_inst(file);
+

+ if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)

+ return -EINVAL;
+
+ switch (s->target) {

+ case V4L2_SEL_TGT_CROP:
+ if (s->r.width != inst->out_width ||
+ s->r.height != inst->out_height ||
+ s->r.top != 0 || s->r.left != 0)
+ return -EINVAL;

+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+

+static int venc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)

+{
+ struct venus_inst *inst = to_inst(file);

+ struct v4l2_outputparm *out = &a->parm.output;
+ struct v4l2_fract *timeperframe = &out->timeperframe;

+ u64 us_per_frame, fps;
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ return -EINVAL;
+

+ memset(out->reserved, 0, sizeof(out->reserved));
+

+ if (!timeperframe->denominator)
+ timeperframe->denominator = inst->timeperframe.denominator;
+ if (!timeperframe->numerator)
+ timeperframe->numerator = inst->timeperframe.numerator;
+

+ out->capability = V4L2_CAP_TIMEPERFRAME;
+

+ us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+ do_div(us_per_frame, timeperframe->denominator);
+
+ if (!us_per_frame)
+ return -EINVAL;
+
+ fps = (u64)USEC_PER_SEC;
+ do_div(fps, us_per_frame);
+

+ inst->timeperframe = *timeperframe;

+ inst->fps = fps;
+

+ return 0;
+}
+

+static int venc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)

+{
+ struct venus_inst *inst = to_inst(file);
+

+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ return -EINVAL;
+

+ a->parm.output.capability |= V4L2_CAP_TIMEPERFRAME;
+ a->parm.output.timeperframe = inst->timeperframe;

+
+ return 0;
+}
+

+static int venc_enum_framesizes(struct file *file, void *fh,

+ struct v4l2_frmsizeenum *fsize)
+{
+ struct venus_inst *inst = to_inst(file);
+ const struct venus_format *fmt;
+

+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+
+ fmt = find_format(fsize->pixel_format,
+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ if (!fmt) {
+ fmt = find_format(fsize->pixel_format,
+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ if (!fmt)
+ return -EINVAL;
+ }
+
+ if (fsize->index)
+ return -EINVAL;
+
+ fsize->stepwise.min_width = inst->cap_width.min;
+ fsize->stepwise.max_width = inst->cap_width.max;
+ fsize->stepwise.step_width = inst->cap_width.step_size;
+ fsize->stepwise.min_height = inst->cap_height.min;
+ fsize->stepwise.max_height = inst->cap_height.max;
+ fsize->stepwise.step_height = inst->cap_height.step_size;
+
+ return 0;
+}
+

+static int venc_enum_frameintervals(struct file *file, void *fh,
+ struct v4l2_frmivalenum *fival)

+{
+ struct venus_inst *inst = to_inst(file);
+ const struct venus_format *fmt;
+

+ fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+
+ fmt = find_format(fival->pixel_format,

+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ if (!fmt) {

+ fmt = find_format(fival->pixel_format,

+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ if (!fmt)
+ return -EINVAL;
+ }
+

+ if (fival->index)
+ return -EINVAL;
+
+ if (!fival->width || !fival->height)
+ return -EINVAL;
+
+ if (fival->width > inst->cap_width.max ||
+ fival->width < inst->cap_width.min ||
+ fival->height > inst->cap_height.max ||
+ fival->height < inst->cap_height.min)
+ return -EINVAL;
+
+ fival->stepwise.min.numerator = 1;
+ fival->stepwise.min.denominator = inst->cap_framerate.max;
+ fival->stepwise.max.numerator = 1;
+ fival->stepwise.max.denominator = inst->cap_framerate.min;
+ fival->stepwise.step.numerator = 1;
+ fival->stepwise.step.denominator = inst->cap_framerate.max;

+
+ return 0;
+}
+

+static const struct v4l2_ioctl_ops venc_ioctl_ops = {
+ .vidioc_querycap = venc_querycap,
+ .vidioc_enum_fmt_vid_cap_mplane = venc_enum_fmt,
+ .vidioc_enum_fmt_vid_out_mplane = venc_enum_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = venc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = venc_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = venc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = venc_g_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = venc_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = venc_try_fmt,
+ .vidioc_g_selection = venc_g_selection,
+ .vidioc_s_selection = venc_s_selection,

+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,

+ .vidioc_s_parm = venc_s_parm,
+ .vidioc_g_parm = venc_g_parm,
+ .vidioc_enum_framesizes = venc_enum_framesizes,
+ .vidioc_enum_frameintervals = venc_enum_frameintervals,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,

+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+

+static int venc_set_properties(struct venus_inst *inst)
+{
+ struct venc_controls *ctr = &inst->controls.enc;
+ struct hfi_intra_period intra_period;
+ struct hfi_profile_level pl;
+ struct hfi_framerate frate;
+ struct hfi_bitrate brate;
+ struct hfi_idr_period idrp;
+ u32 ptype, rate_control, bitrate, profile = 0, level = 0;
+ int ret;
+
+ ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
+ frate.buffer_type = HFI_BUFFER_OUTPUT;
+ frate.framerate = inst->fps * (1 << 16);
+
+ ret = hfi_session_set_property(inst, ptype, &frate);

+ if (ret)
+ return ret;
+

+ if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264) {
+ struct hfi_h264_vui_timing_info info;
+
+ ptype = HFI_PROPERTY_PARAM_VENC_H264_VUI_TIMING_INFO;
+ info.enable = 1;
+ info.fixed_framerate = 1;
+ info.time_scale = NSEC_PER_SEC;
+
+ ret = hfi_session_set_property(inst, ptype, &info);

+ if (ret)
+ return ret;
+ }
+

+ ptype = HFI_PROPERTY_CONFIG_VENC_IDR_PERIOD;
+ idrp.idr_period = ctr->gop_size;
+ ret = hfi_session_set_property(inst, ptype, &idrp);

+ if (ret)
+ return ret;
+

+ if (ctr->num_b_frames) {
+ u32 max_num_b_frames = NUM_B_FRAMES_MAX;
+
+ ptype = HFI_PROPERTY_PARAM_VENC_MAX_NUM_B_FRAMES;
+ ret = hfi_session_set_property(inst, ptype, &max_num_b_frames);

+ if (ret)
+ return ret;
+ }
+

+ /* intra_period = pframes + bframes + 1 */
+ if (!ctr->num_p_frames)
+ ctr->num_p_frames = 2 * 15 - 1,
+
+ ptype = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD;
+ intra_period.pframes = ctr->num_p_frames;
+ intra_period.bframes = ctr->num_b_frames;
+
+ ret = hfi_session_set_property(inst, ptype, &intra_period);

+ if (ret)
+ return ret;
+

+ if (ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR)
+ rate_control = HFI_RATE_CONTROL_VBR_CFR;
+ else
+ rate_control = HFI_RATE_CONTROL_CBR_CFR;
+
+ ptype = HFI_PROPERTY_PARAM_VENC_RATE_CONTROL;
+ ret = hfi_session_set_property(inst, ptype, &rate_control);

+ if (ret)
+ return ret;
+

+ if (!ctr->bitrate)
+ bitrate = 64000;
+ else
+ bitrate = ctr->bitrate;
+
+ ptype = HFI_PROPERTY_CONFIG_VENC_TARGET_BITRATE;
+ brate.bitrate = bitrate;
+ brate.layer_id = 0;
+
+ ret = hfi_session_set_property(inst, ptype, &brate);

+ if (ret)
+ return ret;
+

+ if (!ctr->bitrate_peak)
+ bitrate *= 2;
+ else
+ bitrate = ctr->bitrate_peak;
+
+ ptype = HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE;
+ brate.bitrate = bitrate;
+ brate.layer_id = 0;
+
+ ret = hfi_session_set_property(inst, ptype, &brate);

+ if (ret)
+ return ret;
+

+ if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264) {
+ profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ ctr->profile);
+ level = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ ctr->level);
+ } else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_VP8) {
+ profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_VPX_PROFILE,
+ ctr->profile);
+ level = 0;
+ } else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_MPEG4) {
+ profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+ ctr->profile);
+ level = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+ ctr->level);
+ } else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H263) {
+ profile = 0;
+ level = 0;
+ }
+
+ ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
+ pl.profile = profile;
+ pl.level = level;
+
+ ret = hfi_session_set_property(inst, ptype, &pl);

+ if (ret)
+ return ret;
+

+ return 0;
+}
+

+static int venc_init_session(struct venus_inst *inst)

+{
+ int ret;
+

+ ret = hfi_session_init(inst, inst->fmt_cap->pixfmt);
+ if (ret)

+ return ret;
+
+ ret = helper_set_input_resolution(inst, inst->out_width,
+ inst->out_height);
+ if (ret)
+ goto deinit;
+

+ ret = helper_set_output_resolution(inst, inst->width, inst->height);

+ if (ret)
+ goto deinit;
+

+ ret = helper_set_color_format(inst, inst->fmt_out->pixfmt);
+ if (ret)

+ goto deinit;
+
+ return 0;
+deinit:
+ hfi_session_deinit(inst);
+ return ret;
+}
+

+static int venc_out_num_buffers(struct venus_inst *inst, unsigned int *num)

+{
+ struct hfi_buffer_requirements bufreq;
+ int ret;
+

+ ret = venc_init_session(inst);

+ if (ret)
+ return ret;
+

+ ret = helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);

+
+ *num = bufreq.count_actual;
+
+ hfi_session_deinit(inst);
+
+ return ret;
+}
+

+static int venc_queue_setup(struct vb2_queue *q,

+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[])
+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);

+ unsigned int p, num, min = 4;

+ int ret = 0;
+

+ switch (q->type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ *num_planes = inst->fmt_out->num_planes;
+

+ ret = venc_out_num_buffers(inst, &num);

+ if (ret)
+ break;
+

+ num = max(num, min);

+ *num_buffers = max(*num_buffers, num);

+ inst->num_input_bufs = *num_buffers;
+

+ for (p = 0; p < *num_planes; ++p)

+ sizes[p] = get_framesize_uncompressed(p, inst->width,
+ inst->height);

+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ *num_planes = inst->fmt_cap->num_planes;

+ *num_buffers = max(*num_buffers, min);

+ inst->num_output_bufs = *num_buffers;

+ sizes[0] = get_framesize_compressed(inst->width, inst->height);
+ break;
+ default:

+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+

+static int venc_verify_conf(struct venus_inst *inst)

+{
+ struct hfi_buffer_requirements bufreq;
+ int ret;
+
+ if (!inst->num_input_bufs || !inst->num_output_bufs)
+ return -EINVAL;
+
+ ret = helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+ if (ret)
+ return ret;
+
+ if (inst->num_output_bufs < bufreq.count_actual ||
+ inst->num_output_bufs < bufreq.count_min)
+ return -EINVAL;
+
+ ret = helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
+ if (ret)
+ return ret;
+

+ if (inst->num_input_bufs < bufreq.count_actual ||
+ inst->num_input_bufs < bufreq.count_min)

+ return -EINVAL;
+
+ return 0;
+}
+

+static int venc_start_streaming(struct vb2_queue *q, unsigned int count)

+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);

+ int ret;
+
+ mutex_lock(&inst->lock);
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ inst->streamon_out = 1;
+ else
+ inst->streamon_cap = 1;
+
+ if (!(inst->streamon_out & inst->streamon_cap)) {
+ mutex_unlock(&inst->lock);
+ return 0;
+ }
+

+ inst->sequence = 0;
+
+ ret = venc_init_session(inst);

+ if (ret)
+ goto bufs_done;
+

+ ret = venc_set_properties(inst);

+ if (ret)
+ goto deinit_sess;
+

+ ret = venc_verify_conf(inst);

+static const struct vb2_ops venc_vb2_ops = {
+ .queue_setup = venc_queue_setup,

+ .buf_init = helper_vb2_buf_init,
+ .buf_prepare = helper_vb2_buf_prepare,

+ .start_streaming = venc_start_streaming,

+ .stop_streaming = helper_vb2_stop_streaming,
+ .buf_queue = helper_vb2_buf_queue,
+};
+

+static void venc_buf_done(struct venus_inst *inst, unsigned int buf_type,

+ u32 tag, u32 bytesused, u32 data_offset, u32 flags,
+ u64 timestamp_us)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct vb2_buffer *vb;
+ unsigned int type;
+
+ if (buf_type == HFI_BUFFER_INPUT)
+ type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ else
+ type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+
+ vbuf = helper_find_buf(inst, type, tag);
+ if (!vbuf)
+ return;
+

+ vb = &vbuf->vb2_buf;

+ vb->planes[0].bytesused = bytesused;

+ vb->planes[0].data_offset = data_offset;
+

+ vbuf->flags = flags;
+
+ if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {

+ vb->timestamp = timestamp_us * NSEC_PER_USEC;
+ vbuf->sequence = inst->sequence++;
+ }
+

+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
+}
+

+static void venc_event_notify(struct venus_inst *inst, u32 event,

+ struct hfi_event_data *data)
+{

+ struct device *dev = inst->core->dev_enc;
+
+ if (event == EVT_SESSION_ERROR) {

+ inst->session_error = true;

+ dev_err(dev, "enc: event session error %x\n", inst->error);
+ }
+}
+
+static const struct hfi_inst_ops venc_hfi_ops = {
+ .buf_done = venc_buf_done,
+ .event_notify = venc_event_notify,
+};
+
+static const struct v4l2_m2m_ops venc_m2m_ops = {

+ .device_run = helper_m2m_device_run,
+ .job_abort = helper_m2m_job_abort,
+};
+
+static int m2m_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct venus_inst *inst = priv;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;

+ src_vq->ops = &venc_vb2_ops;

+ src_vq->mem_ops = &vb2_dma_sg_memops;
+ src_vq->drv_priv = inst;
+ src_vq->buf_struct_size = sizeof(struct venus_buffer);
+ src_vq->allow_zero_bytesused = 1;
+ src_vq->min_buffers_needed = 1;
+ src_vq->dev = inst->core->dev;
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;

+ dst_vq->ops = &venc_vb2_ops;

+ dst_vq->mem_ops = &vb2_dma_sg_memops;
+ dst_vq->drv_priv = inst;
+ dst_vq->buf_struct_size = sizeof(struct venus_buffer);
+ dst_vq->allow_zero_bytesused = 1;
+ dst_vq->min_buffers_needed = 1;
+ dst_vq->dev = inst->core->dev;
+ ret = vb2_queue_init(dst_vq);
+ if (ret) {
+ vb2_queue_release(src_vq);
+ return ret;
+ }
+
+ return 0;
+}
+

+static void venc_inst_init(struct venus_inst *inst)
+{
+ inst->fmt_cap = &venc_formats[2];
+ inst->fmt_out = &venc_formats[0];

+ inst->width = 1280;
+ inst->height = ALIGN(720, 32);

+ inst->fps = 15;

+ inst->timeperframe.numerator = 1;

+ inst->timeperframe.denominator = 15;

+
+ inst->cap_width.min = 64;

+ inst->cap_width.max = 1920;

+ inst->cap_width.step_size = 1;
+ inst->cap_height.min = 64;

+ inst->cap_height.max = ALIGN(1080, 32);

+ inst->cap_height.step_size = 1;
+ inst->cap_framerate.min = 1;
+ inst->cap_framerate.max = 30;
+ inst->cap_framerate.step_size = 1;
+ inst->cap_mbs_per_frame.min = 16;
+ inst->cap_mbs_per_frame.max = 8160;
+}
+

+static int venc_open(struct file *file)

+{
+ struct venus_core *core = video_drvdata(file);
+ struct venus_inst *inst;
+ int ret;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&inst->registeredbufs);
+ INIT_LIST_HEAD(&inst->internalbufs);
+ INIT_LIST_HEAD(&inst->list);
+ mutex_init(&inst->lock);
+
+ inst->core = core;

+ inst->session_type = VIDC_SESSION_TYPE_ENC;
+
+ ret = pm_runtime_get_sync(core->dev_enc);

+ if (ret < 0)
+ goto err_free_inst;
+

+ ret = venc_ctrl_init(inst);

+ if (ret)
+ goto err_put_sync;
+

+ ret = hfi_session_create(inst, &venc_hfi_ops);

+ if (ret)
+ goto err_ctrl_deinit;
+

+ venc_inst_init(inst);

+
+ /*
+ * create m2m device for every instance, the m2m context scheduling
+ * is made by firmware side so we do not need to care about.
+ */

+ inst->m2m_dev = v4l2_m2m_init(&venc_m2m_ops);

+ if (IS_ERR(inst->m2m_dev)) {
+ ret = PTR_ERR(inst->m2m_dev);
+ goto err_session_destroy;
+ }
+
+ inst->m2m_ctx = v4l2_m2m_ctx_init(inst->m2m_dev, inst, m2m_queue_init);
+ if (IS_ERR(inst->m2m_ctx)) {
+ ret = PTR_ERR(inst->m2m_ctx);
+ goto err_m2m_release;
+ }
+

+ v4l2_fh_init(&inst->fh, core->vdev_enc);

+
+ inst->fh.ctrl_handler = &inst->ctrl_handler;
+ v4l2_fh_add(&inst->fh);
+ inst->fh.m2m_ctx = inst->m2m_ctx;
+ file->private_data = &inst->fh;
+
+ return 0;
+
+err_m2m_release:
+ v4l2_m2m_release(inst->m2m_dev);
+err_session_destroy:
+ hfi_session_destroy(inst);
+err_ctrl_deinit:

+ venc_ctrl_deinit(inst);
+err_put_sync:
+ pm_runtime_put_sync(core->dev_enc);

+err_free_inst:
+ kfree(inst);
+ return ret;
+}
+

+static int venc_close(struct file *file)

+{
+ struct venus_inst *inst = to_inst(file);
+
+ v4l2_m2m_ctx_release(inst->m2m_ctx);
+ v4l2_m2m_release(inst->m2m_dev);

+ venc_ctrl_deinit(inst);

+ hfi_session_destroy(inst);
+ mutex_destroy(&inst->lock);
+ v4l2_fh_del(&inst->fh);
+ v4l2_fh_exit(&inst->fh);
+ kfree(inst);
+

+ pm_runtime_put_sync(inst->core->dev_enc);

+
+ return 0;
+}
+

+static const struct v4l2_file_operations venc_fops = {
+ .owner = THIS_MODULE,
+ .open = venc_open,
+ .release = venc_close,

+ .unlocked_ioctl = video_ioctl2,
+ .poll = v4l2_m2m_fop_poll,
+ .mmap = v4l2_m2m_fop_mmap,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+

+static int venc_probe(struct platform_device *pdev)

+{
+ struct device *dev = &pdev->dev;
+ struct video_device *vdev;
+ struct venus_core *core;
+ int ret;
+
+ if (!dev->parent)
+ return -EPROBE_DEFER;
+
+ core = dev_get_drvdata(dev->parent);
+ if (!core)
+ return -EPROBE_DEFER;
+
+ if (core->res->hfi_version == HFI_VERSION_3XX) {

+ core->core1_clk = devm_clk_get(dev, "core");
+ if (IS_ERR(core->core1_clk))
+ return PTR_ERR(core->core1_clk);

+ }
+
+ platform_set_drvdata(pdev, core);
+
+ vdev = video_device_alloc();
+ if (!vdev)
+ return -ENOMEM;
+
+ vdev->release = video_device_release;

+ vdev->fops = &venc_fops;
+ vdev->ioctl_ops = &venc_ioctl_ops;

+ vdev->vfl_dir = VFL_DIR_M2M;
+ vdev->v4l2_dev = &core->v4l2_dev;
+ vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+
+ ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+ if (ret)
+ goto err_vdev_release;
+

+ core->vdev_enc = vdev;
+ core->dev_enc = dev;

+
+ video_set_drvdata(vdev, core);
+ pm_runtime_enable(dev);
+
+ return 0;
+
+err_vdev_release:
+ video_device_release(vdev);
+ return ret;
+}
+

+static int venc_remove(struct platform_device *pdev)

+{
+ struct venus_core *core = dev_get_drvdata(pdev->dev.parent);
+

+ video_unregister_device(core->vdev_enc);
+ pm_runtime_disable(core->dev_enc);

+
+ return 0;
+}
+
+#ifdef CONFIG_PM

+static int venc_runtime_suspend(struct device *dev)

+{
+ struct venus_core *core = dev_get_drvdata(dev);
+
+ if (core->res->hfi_version == HFI_VERSION_LEGACY)
+ return 0;
+

+ writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+ clk_disable_unprepare(core->core1_clk);
+ writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);

+
+ return 0;
+}
+

+static int venc_runtime_resume(struct device *dev)

+{
+ struct venus_core *core = dev_get_drvdata(dev);
+ int ret;
+
+ if (core->res->hfi_version == HFI_VERSION_LEGACY)
+ return 0;
+

+ writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+ ret = clk_prepare_enable(core->core1_clk);
+ writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);

+
+ return ret;
+}
+#endif
+

+static const struct dev_pm_ops venc_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(venc_runtime_suspend, venc_runtime_resume, NULL)
+};
+
+static const struct of_device_id venc_dt_match[] = {
+ { .compatible = "venus-encoder" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, venc_dt_match);
+
+static struct platform_driver qcom_venus_enc_driver = {
+ .probe = venc_probe,
+ .remove = venc_remove,
+ .driver = {
+ .name = "qcom-venus-encoder",
+ .of_match_table = venc_dt_match,
+ .pm = &venc_pm_ops,
+ },
+};
+module_platform_driver(qcom_venus_enc_driver);
+
+MODULE_ALIAS("platform:qcom-venus-encoder");
+MODULE_DESCRIPTION("Qualcomm Venus video encoder driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/venus/venc.h
new file mode 100644
index 000000000000..9daca669f307
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/venc.h

@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */

+#ifndef __VENUS_VENC_H__
+#define __VENUS_VENC_H__
+
+struct venus_inst;
+
+int venc_ctrl_init(struct venus_inst *inst);
+void venc_ctrl_deinit(struct venus_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/venus/venc_ctrls.c
new file mode 100644
index 000000000000..0d2eecea7edb
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/venc_ctrls.c
@@ -0,0 +1,258 @@

+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+
+#include "core.h"
+

+#define BITRATE_MIN 32000
+#define BITRATE_MAX 160000000
+#define BITRATE_DEFAULT 1000000
+#define BITRATE_DEFAULT_PEAK (BITRATE_DEFAULT * 2)
+#define BITRATE_STEP 100
+#define SLICE_BYTE_SIZE_MAX 1024
+#define SLICE_BYTE_SIZE_MIN 1024
+#define SLICE_MB_SIZE_MAX 300
+#define INTRA_REFRESH_MBS_MAX 300
+#define AT_SLICE_BOUNDARY \
+ V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY
+
+static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl)

+{
+ struct venus_inst *inst = ctrl_to_inst(ctrl);

+ struct venc_controls *ctr = &inst->controls.enc;

+
+ switch (ctrl->id) {

+ case V4L2_CID_MPEG_VIDEO_BITRATE_MODE:
+ ctr->bitrate_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_BITRATE:
+ ctr->bitrate = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_BITRATE_PEAK:
+ ctr->bitrate_peak = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
+ ctr->h264_entropy_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+ case V4L2_CID_MPEG_VIDEO_H264_PROFILE:

+ case V4L2_CID_MPEG_VIDEO_VPX_PROFILE:
+ ctr->profile = ctrl->val;
+ break;

+ case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+ case V4L2_CID_MPEG_VIDEO_H264_LEVEL:

+ ctr->level = ctrl->val;
+ break;

+ case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
+ ctr->h264_i_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
+ ctr->h264_p_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP:
+ ctr->h264_b_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_MIN_QP:
+ ctr->h264_min_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+ ctr->h264_max_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE:
+ ctr->multi_slice_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES:
+ ctr->multi_slice_max_bytes = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB:
+ ctr->multi_slice_max_mb = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA:
+ ctr->h264_loop_filter_alpha = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA:
+ ctr->h264_loop_filter_beta = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE:
+ ctr->h264_loop_filter_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
+ ctr->header_mode = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB:
+ break;
+ case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+ ctr->gop_size = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_I_PERIOD:
+ ctr->h264_i_period = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_VPX_MIN_QP:
+ ctr->vp8_min_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_VPX_MAX_QP:
+ ctr->vp8_max_qp = ctrl->val;
+ break;
+ case V4L2_CID_MPEG_VIDEO_B_FRAMES:
+ ctr->num_b_frames = ctrl->val;

+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+

+static const struct v4l2_ctrl_ops venc_ctrl_ops = {
+ .s_ctrl = venc_op_s_ctrl,
+};
+
+int venc_ctrl_init(struct venus_inst *inst)

+{
+ int ret;
+

+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 27);

+ if (ret)
+ return ret;
+

+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ ~((1 << V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) |
+ (1 << V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)),
+ V4L2_MPEG_VIDEO_BITRATE_MODE_VBR);
+
+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+ 0, V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC);
+
+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,

+ V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE,
+ V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_CODING_EFFICIENCY,
+ ~((1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE) |
+ (1 << V4L2_MPEG_VIDEO_MPEG4_PROFILE_ADVANCED_SIMPLE)),
+ V4L2_MPEG_VIDEO_MPEG4_PROFILE_SIMPLE);
+

+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,

+ V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL,
+ V4L2_MPEG_VIDEO_MPEG4_LEVEL_5,
+ 0, V4L2_MPEG_VIDEO_MPEG4_LEVEL_0);
+

+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,

+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH,
+ ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH) |
+ (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH)),

+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH);
+
+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,

+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_1,
+ 0, V4L2_MPEG_VIDEO_H264_LEVEL_1_0);
+

+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE,
+ AT_SLICE_BOUNDARY,
+ 0, V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED);
+
+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+ V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ 1 << V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE);
+
+ v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE,
+ V4L2_MPEG_VIDEO_MULTI_SICE_MODE_MAX_BYTES,
+ 0, V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_BITRATE, BITRATE_MIN, BITRATE_MAX,
+ BITRATE_STEP, BITRATE_DEFAULT);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_BITRATE_PEAK, BITRATE_MIN, BITRATE_MAX,
+ BITRATE_STEP, BITRATE_DEFAULT_PEAK);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,

+ V4L2_CID_MPEG_VIDEO_VPX_PROFILE, 0, 3, 1, 0);
+

+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 1, 51, 1, 26);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 1, 51, 1, 28);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP, 1, 51, 1, 30);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_MIN_QP, 1, 51, 1, 1);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_MAX_QP, 1, 51, 1, 51);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES, SLICE_BYTE_SIZE_MIN,
+ SLICE_BYTE_SIZE_MAX, 1, SLICE_BYTE_SIZE_MIN);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB, 1,
+ SLICE_MB_SIZE_MAX, 1, 1);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA, -6, 6, 1, 0);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA, -6, 6, 1, 0);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB,
+ 0, INTRA_REFRESH_MBS_MAX, 1, 0);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_GOP_SIZE, 0, (1 << 16) - 1, 1, 12);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_VPX_MIN_QP, 1, 128, 1, 1);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_VPX_MAX_QP, 1, 128, 1, 128);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_B_FRAMES, 0, 4, 1, 0);
+
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_I_PERIOD, 0, (1 << 16) - 1, 1, 0);

+
+ ret = inst->ctrl_handler.error;
+ if (ret) {
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
+ return ret;
+ }
+
+ return 0;
+}
+

+void venc_ctrl_deinit(struct venus_inst *inst)

[Stanimir Varbanov]

This adds Venus driver Makefile and changes v4l2 platform
Makefile/Kconfig in order to enable building of the driver.

Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
---

drivers/media/platform/Kconfig | 14 ++++++++++++++
drivers/media/platform/Makefile | 2 ++
drivers/media/platform/qcom/venus/Makefile | 11 +++++++++++
3 files changed, 27 insertions(+)
create mode 100644 drivers/media/platform/qcom/venus/Makefile

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 53f6f12bff0d..8a6c3d664307 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -447,6 +447,20 @@ config VIDEO_TI_VPE_DEBUG
---help---
Enable debug messages on VPE driver.

+config VIDEO_QCOM_VENUS
+ tristate "Qualcomm Venus V4L2 encoder/decoder driver"
+ depends on VIDEO_DEV && VIDEO_V4L2 && HAS_DMA
+ depends on ARCH_QCOM && OF
+ depends on IOMMU_DMA
+ select QCOM_MDT_LOADER
+ select VIDEOBUF2_DMA_SG
+ select V4L2_MEM2MEM_DEV
+ ---help---
+ This is a V4L2 driver for Qualcomm Venus video accelerator
+ hardware. It accelerates encoding and decoding operations
+ on various Qualcomm SoCs.
+ To compile this driver as a module choose m here.
+
endif # V4L_MEM2MEM_DRIVERS

# TI VIDEO PORT Helper Modules
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 8959f6e6692a..bd5cae68db8a 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -73,3 +73,5 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec/
obj-$(CONFIG_VIDEO_MEDIATEK_MDP) += mtk-mdp/

obj-$(CONFIG_VIDEO_MEDIATEK_JPEG) += mtk-jpeg/
+
+obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/
diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/venus/Makefile
new file mode 100644
index 000000000000..0fe9afb83697
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/Makefile
@@ -0,0 +1,11 @@
+# Makefile for Qualcomm Venus driver
+
+venus-core-objs += core.o helpers.o firmware.o \
+ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o
+
+venus-dec-objs += vdec.o vdec_ctrls.o
+venus-enc-objs += venc.o venc_ctrls.o
+
+obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-core.o
+obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-dec.o
+obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-enc.o
--
2.7.4

[Stanimir Varbanov]

Here is the implementation of Venus video accelerator low-level
functionality. It contanins code which setup the registers and
startup uthe processor, allocate and manipulates with the shared
memory used for sending commands and receiving messages.

Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
---

drivers/media/platform/qcom/venus/hfi_venus.c | 1570 ++++++++++++++++++++++

drivers/media/platform/qcom/venus/hfi_venus.h | 23 +
drivers/media/platform/qcom/venus/hfi_venus_io.h | 113 ++

3 files changed, 1706 insertions(+)

create mode 100644 drivers/media/platform/qcom/venus/hfi_venus.c
create mode 100644 drivers/media/platform/qcom/venus/hfi_venus.h
create mode 100644 drivers/media/platform/qcom/venus/hfi_venus_io.h

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
new file mode 100644
index 000000000000..74533359d12f
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -0,0 +1,1570 @@

+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+

+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/qcom_scm.h>
+#include <linux/slab.h>
+
+#include "core.h"
+#include "hfi_cmds.h"
+#include "hfi_msgs.h"
+#include "hfi_venus.h"
+#include "hfi_venus_io.h"
+
+#define HFI_MASK_QHDR_TX_TYPE 0xff000000
+#define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
+#define HFI_MASK_QHDR_PRI_TYPE 0x0000ff00
+#define HFI_MASK_QHDR_ID_TYPE 0x000000ff
+
+#define HFI_HOST_TO_CTRL_CMD_Q 0
+#define HFI_CTRL_TO_HOST_MSG_Q 1
+#define HFI_CTRL_TO_HOST_DBG_Q 2
+#define HFI_MASK_QHDR_STATUS 0x000000ff
+
+#define IFACEQ_NUM 3
+#define IFACEQ_CMD_IDX 0
+#define IFACEQ_MSG_IDX 1
+#define IFACEQ_DBG_IDX 2
+#define IFACEQ_MAX_BUF_COUNT 50
+#define IFACEQ_MAX_PARALLEL_CLNTS 16
+#define IFACEQ_DFLT_QHDR 0x01010000
+
+#define POLL_INTERVAL_US 50
+
+#define IFACEQ_MAX_PKT_SIZE 1024
+#define IFACEQ_MED_PKT_SIZE 768
+#define IFACEQ_MIN_PKT_SIZE 8
+#define IFACEQ_VAR_SMALL_PKT_SIZE 100
+#define IFACEQ_VAR_LARGE_PKT_SIZE 512
+#define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12)
+
+enum tzbsp_video_state {
+ TZBSP_VIDEO_STATE_SUSPEND = 0,
+ TZBSP_VIDEO_STATE_RESUME
+};
+
+struct hfi_queue_table_header {
+ u32 version;
+ u32 size;
+ u32 qhdr0_offset;
+ u32 qhdr_size;
+ u32 num_q;
+ u32 num_active_q;
+};
+
+struct hfi_queue_header {
+ u32 status;
+ u32 start_addr;
+ u32 type;
+ u32 q_size;
+ u32 pkt_size;
+ u32 pkt_drop_cnt;
+ u32 rx_wm;
+ u32 tx_wm;
+ u32 rx_req;
+ u32 tx_req;
+ u32 rx_irq_status;
+ u32 tx_irq_status;
+ u32 read_idx;
+ u32 write_idx;
+};
+
+#define IFACEQ_TABLE_SIZE \
+ (sizeof(struct hfi_queue_table_header) + \
+ sizeof(struct hfi_queue_header) * IFACEQ_NUM)
+
+#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \
+ IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS)
+
+#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \
+ (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \
+ ((i) * sizeof(struct hfi_queue_header)))
+
+#define QDSS_SIZE SZ_4K
+#define SFR_SIZE SZ_4K
+#define QUEUE_SIZE \
+ (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM))
+
+#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K)
+#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K)
+#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K)
+#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \
+ ALIGNED_QDSS_SIZE, SZ_1M)
+
+struct mem_desc {
+ dma_addr_t da; /* device address */
+ void *kva; /* kernel virtual address */
+ u32 size;
+ unsigned long attrs;
+};
+
+struct iface_queue {
+ struct hfi_queue_header *qhdr;
+ struct mem_desc qmem;
+};
+
+enum venus_state {
+ VENUS_STATE_DEINIT = 1,
+ VENUS_STATE_INIT,
+};
+
+struct venus_hfi_device {
+ struct venus_core *core;
+ u32 irq_status;
+ u32 last_packet_type;
+ bool power_enabled;
+ bool suspended;
+ enum venus_state state;
+ struct mutex lock;
+ struct completion pwr_collapse_prep;
+ struct completion release_resource;
+ struct mem_desc ifaceq_table;
+ struct mem_desc sfr;
+ struct iface_queue queues[IFACEQ_NUM];
+ u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
+ u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE];
+};
+
+static bool venus_pkt_debug;
+static int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL;
+static bool venus_sys_idle_indicator;
+static bool venus_fw_low_power_mode = true;
+static int venus_hw_rsp_timeout = 1000;
+static bool venus_fw_coverage;
+
+static void venus_set_state(struct venus_hfi_device *hdev,
+ enum venus_state state)
+{
+ mutex_lock(&hdev->lock);
+ hdev->state = state;
+ mutex_unlock(&hdev->lock);
+}
+
+static bool venus_is_valid_state(struct venus_hfi_device *hdev)
+{
+ return hdev->state != VENUS_STATE_DEINIT;
+}
+
+static void venus_dump_packet(struct venus_hfi_device *hdev, const void *packet)
+{
+ size_t pkt_size = *(u32 *)packet;
+
+ if (!venus_pkt_debug)
+ return;
+
+ print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet,
+ pkt_size, true);
+}
+
+static int venus_write_queue(struct venus_hfi_device *hdev,
+ struct iface_queue *queue,
+ void *packet, u32 *rx_req)
+{
+ struct hfi_queue_header *qhdr;
+ u32 dwords, new_wr_idx;
+ u32 empty_space, rd_idx, wr_idx, qsize;
+ u32 *wr_ptr;
+
+ if (!queue->qmem.kva)
+ return -EINVAL;
+
+ qhdr = queue->qhdr;
+ if (!qhdr)
+ return -EINVAL;
+
+ venus_dump_packet(hdev, packet);
+
+ dwords = (*(u32 *)packet) >> 2;
+ if (!dwords)
+ return -EINVAL;
+
+ rd_idx = qhdr->read_idx;
+ wr_idx = qhdr->write_idx;
+ qsize = qhdr->q_size;
+ /* ensure rd/wr indices's are read from memory */
+ rmb();
+
+ if (wr_idx >= rd_idx)
+ empty_space = qsize - (wr_idx - rd_idx);
+ else
+ empty_space = rd_idx - wr_idx;
+
+ if (empty_space <= dwords) {
+ qhdr->tx_req = 1;
+ /* ensure tx_req is updated in memory */
+ wmb();
+ return -ENOSPC;
+ }
+
+ qhdr->tx_req = 0;
+ /* ensure tx_req is updated in memory */
+ wmb();
+
+ new_wr_idx = wr_idx + dwords;
+ wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2));
+ if (new_wr_idx < qsize) {
+ memcpy(wr_ptr, packet, dwords << 2);
+ } else {
+ size_t len;
+
+ new_wr_idx -= qsize;
+ len = (dwords - new_wr_idx) << 2;
+ memcpy(wr_ptr, packet, len);
+ memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2);
+ }
+
+ /* make sure packet is written before updating the write index */
+ wmb();
+
+ qhdr->write_idx = new_wr_idx;
+ *rx_req = qhdr->rx_req ? 1 : 0;
+
+ /* make sure write index is updated before an interrupt is raised */
+ mb();

+
+ return 0;
+}
+

+static int venus_read_queue(struct venus_hfi_device *hdev,
+ struct iface_queue *queue, void *pkt, u32 *tx_req)
+{
+ struct hfi_queue_header *qhdr;
+ u32 dwords, new_rd_idx;
+ u32 rd_idx, wr_idx, type, qsize;
+ u32 *rd_ptr;
+ u32 recv_request = 0;
+ int ret = 0;
+
+ if (!queue->qmem.kva)
+ return -EINVAL;
+
+ qhdr = queue->qhdr;
+ if (!qhdr)
+ return -EINVAL;
+
+ type = qhdr->type;
+ rd_idx = qhdr->read_idx;
+ wr_idx = qhdr->write_idx;
+ qsize = qhdr->q_size;
+
+ /* make sure data is valid before using it */
+ rmb();
+
+ /*
+ * Do not set receive request for debug queue, if set, Venus generates
+ * interrupt for debug messages even when there is no response message
+ * available. In general debug queue will not become full as it is being
+ * emptied out for every interrupt from Venus. Venus will anyway
+ * generates interrupt if it is full.
+ */
+ if (type & HFI_CTRL_TO_HOST_MSG_Q)
+ recv_request = 1;
+
+ if (rd_idx == wr_idx) {
+ qhdr->rx_req = recv_request;
+ *tx_req = 0;
+ /* update rx_req field in memory */
+ wmb();
+ return -ENODATA;
+ }
+
+ rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2));
+ dwords = *rd_ptr >> 2;
+ if (!dwords)
+ return -EINVAL;
+
+ new_rd_idx = rd_idx + dwords;
+ if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) {
+ if (new_rd_idx < qsize) {
+ memcpy(pkt, rd_ptr, dwords << 2);
+ } else {
+ size_t len;
+
+ new_rd_idx -= qsize;
+ len = (dwords - new_rd_idx) << 2;
+ memcpy(pkt, rd_ptr, len);
+ memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2);
+ }
+ } else {
+ /* bad packet received, dropping */
+ new_rd_idx = qhdr->write_idx;
+ ret = -EBADMSG;
+ }
+
+ /* ensure the packet is read before updating read index */
+ rmb();
+
+ qhdr->read_idx = new_rd_idx;
+ /* ensure updating read index */
+ wmb();
+
+ rd_idx = qhdr->read_idx;
+ wr_idx = qhdr->write_idx;
+ /* ensure rd/wr indices are read from memory */
+ rmb();
+
+ if (rd_idx != wr_idx)
+ qhdr->rx_req = 0;
+ else
+ qhdr->rx_req = recv_request;
+
+ *tx_req = qhdr->tx_req ? 1 : 0;
+
+ /* ensure rx_req is stored to memory and tx_req is loaded from memory */
+ mb();
+
+ venus_dump_packet(hdev, pkt);

+
+ return ret;
+}
+

+static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc,
+ u32 size)
+{
+ struct device *dev = hdev->core->dev;
+
+ desc->attrs = DMA_ATTR_WRITE_COMBINE;
+ desc->size = ALIGN(size, SZ_4K);
+
+ desc->kva = dma_alloc_attrs(dev, size, &desc->da, GFP_KERNEL,
+ desc->attrs);
+ if (!desc->kva)
+ return -ENOMEM;
+

+ return 0;
+}
+

+static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem)
+{
+ struct device *dev = hdev->core->dev;
+
+ dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs);
+}
+
+static void venus_writel(struct venus_hfi_device *hdev, u32 reg, u32 value)
+{
+ writel(value, hdev->core->base + reg);
+}
+
+static u32 venus_readl(struct venus_hfi_device *hdev, u32 reg)
+{
+ return readl(hdev->core->base + reg);
+}
+
+static void venus_set_registers(struct venus_hfi_device *hdev)
+{
+ const struct venus_resources *res = hdev->core->res;
+ const struct reg_val *tbl = res->reg_tbl;
+ unsigned int count = res->reg_tbl_size;

+ unsigned int i;
+

+ for (i = 0; i < count; i++)
+ venus_writel(hdev, tbl[i].reg, tbl[i].value);
+}
+
+static void venus_soft_int(struct venus_hfi_device *hdev)
+{
+ venus_writel(hdev, CPU_IC_SOFTINT, BIT(CPU_IC_SOFTINT_H2A_SHIFT));
+}
+
+static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
+ void *pkt)
+{
+ struct device *dev = hdev->core->dev;
+ struct hfi_pkt_hdr *cmd_packet;
+ struct iface_queue *queue;
+ u32 rx_req;
+ int ret;
+
+ if (!venus_is_valid_state(hdev))
+ return -EINVAL;
+
+ cmd_packet = (struct hfi_pkt_hdr *)pkt;
+ hdev->last_packet_type = cmd_packet->pkt_type;
+
+ queue = &hdev->queues[IFACEQ_CMD_IDX];
+
+ ret = venus_write_queue(hdev, queue, pkt, &rx_req);
+ if (ret) {
+ dev_err(dev, "write to iface cmd queue failed (%d)\n", ret);

+ return ret;
+ }
+

+ if (rx_req)
+ venus_soft_int(hdev);

+
+ return 0;
+}
+

+static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt)

+{
+ int ret;
+

+ mutex_lock(&hdev->lock);
+ ret = venus_iface_cmdq_write_nolock(hdev, pkt);
+ mutex_unlock(&hdev->lock);

+
+ return ret;
+}
+

+static int venus_hfi_core_set_resource(struct venus_core *core, u32 id,
+ u32 size, u32 addr, void *cookie)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct hfi_sys_set_resource_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ if (id == VIDC_RESOURCE_NONE)
+ return 0;
+
+ pkt = (struct hfi_sys_set_resource_pkt *) packet;
+
+ ret = pkt_sys_set_resource(pkt, id, size, addr, cookie);

+ if (ret)
+ return ret;
+

+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_boot_core(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ static const unsigned int max_tries = 100;
+ u32 ctrl_status = 0;
+ unsigned int count = 0;
+ int ret = 0;
+
+ venus_writel(hdev, VIDC_CTRL_INIT, BIT(VIDC_CTRL_INIT_CTRL_SHIFT));
+ venus_writel(hdev, WRAPPER_INTR_MASK, WRAPPER_INTR_MASK_A2HVCODEC_MASK);
+ venus_writel(hdev, CPU_CS_SCIACMDARG3, 1);
+
+ while (!ctrl_status && count < max_tries) {
+ ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+ if ((ctrl_status & CPU_CS_SCIACMDARG0_ERROR_STATUS_MASK) == 4) {
+ dev_err(dev, "invalid setting for UC_REGION\n");

+ ret = -EINVAL;
+ break;
+ }
+

+ usleep_range(500, 1000);
+ count++;
+ }
+
+ if (count >= max_tries)
+ ret = -ETIMEDOUT;

+
+ return ret;
+}
+

+static u32 venus_hwversion(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ u32 ver = venus_readl(hdev, WRAPPER_HW_VERSION);
+ u32 major, minor, step;
+
+ major = ver & WRAPPER_HW_VERSION_MAJOR_VERSION_MASK;
+ major = major >> WRAPPER_HW_VERSION_MAJOR_VERSION_SHIFT;
+ minor = ver & WRAPPER_HW_VERSION_MINOR_VERSION_MASK;
+ minor = minor >> WRAPPER_HW_VERSION_MINOR_VERSION_SHIFT;
+ step = ver & WRAPPER_HW_VERSION_STEP_VERSION_MASK;
+
+ dev_dbg(dev, "venus hw version %x.%x.%x\n", major, minor, step);
+
+ return major;
+}
+
+static int venus_run(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ int ret;
+
+ /*
+ * Re-program all of the registers that get reset as a result of
+ * regulator_disable() and _enable()
+ */
+ venus_set_registers(hdev);
+
+ venus_writel(hdev, UC_REGION_ADDR, hdev->ifaceq_table.da);
+ venus_writel(hdev, UC_REGION_SIZE, SHARED_QSIZE);
+ venus_writel(hdev, CPU_CS_SCIACMDARG2, hdev->ifaceq_table.da);
+ venus_writel(hdev, CPU_CS_SCIACMDARG1, 0x01);
+ if (hdev->sfr.da)
+ venus_writel(hdev, SFR_ADDR, hdev->sfr.da);
+
+ ret = venus_boot_core(hdev);
+ if (ret) {
+ dev_err(dev, "failed to reset venus core\n");

+ return ret;
+ }
+

+ venus_hwversion(hdev);

+
+ return 0;
+}
+

+static int venus_halt_axi(struct venus_hfi_device *hdev)
+{
+ void __iomem *base = hdev->core->base;
+ struct device *dev = hdev->core->dev;
+ u32 val;
+ int ret;
+
+ /* Halt AXI and AXI IMEM VBIF Access */
+ val = venus_readl(hdev, VBIF_AXI_HALT_CTRL0);
+ val |= VBIF_AXI_HALT_CTRL0_HALT_REQ;
+ venus_writel(hdev, VBIF_AXI_HALT_CTRL0, val);
+
+ /* Request for AXI bus port halt */
+ ret = readl_poll_timeout(base + VBIF_AXI_HALT_CTRL1, val,
+ val & VBIF_AXI_HALT_CTRL1_HALT_ACK,
+ POLL_INTERVAL_US,
+ VBIF_AXI_HALT_ACK_TIMEOUT_US);
+ if (ret) {
+ dev_err(dev, "AXI bus port halt timeout\n");

+ return ret;
+ }
+
+ return 0;
+}
+

+static int venus_power_off(struct venus_hfi_device *hdev)

+{
+ int ret;
+

+ if (!hdev->power_enabled)
+ return 0;
+
+ ret = qcom_scm_set_remote_state(TZBSP_VIDEO_STATE_SUSPEND, 0);

+ if (ret)
+ return ret;
+

+ ret = venus_halt_axi(hdev);

+ if (ret)
+ return ret;
+

+ hdev->power_enabled = false;

+
+ return 0;
+}
+

+static int venus_power_on(struct venus_hfi_device *hdev)

+{
+ int ret;
+

+ if (hdev->power_enabled)
+ return 0;
+
+ ret = qcom_scm_set_remote_state(TZBSP_VIDEO_STATE_RESUME, 0);
+ if (ret)
+ goto err;
+
+ ret = venus_run(hdev);
+ if (ret)
+ goto err_suspend;
+
+ hdev->power_enabled = true;

+
+ return 0;
+

+err_suspend:
+ qcom_scm_set_remote_state(TZBSP_VIDEO_STATE_SUSPEND, 0);
+err:
+ hdev->power_enabled = false;

+ return ret;
+}
+

+static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev,
+ void *pkt)
+{
+ struct iface_queue *queue;
+ u32 tx_req;
+ int ret;
+
+ if (!venus_is_valid_state(hdev))
+ return -EINVAL;
+
+ queue = &hdev->queues[IFACEQ_MSG_IDX];
+
+ ret = venus_read_queue(hdev, queue, pkt, &tx_req);

+ if (ret)
+ return ret;
+

+ if (tx_req)
+ venus_soft_int(hdev);

+
+ return 0;
+}
+

+static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt)

+{
+ int ret;
+

+ mutex_lock(&hdev->lock);
+ ret = venus_iface_msgq_read_nolock(hdev, pkt);
+ mutex_unlock(&hdev->lock);

+
+ return ret;
+}
+

+static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev,
+ void *pkt)
+{
+ struct iface_queue *queue;
+ u32 tx_req;
+ int ret;
+
+ ret = venus_is_valid_state(hdev);
+ if (!ret)
+ return -EINVAL;
+
+ queue = &hdev->queues[IFACEQ_DBG_IDX];
+
+ ret = venus_read_queue(hdev, queue, pkt, &tx_req);

+ if (ret)
+ return ret;
+

+ if (tx_req)
+ venus_soft_int(hdev);

+
+ return 0;
+}
+

+static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt)

+{
+ int ret;
+

+ if (!pkt)
+ return -EINVAL;
+
+ mutex_lock(&hdev->lock);
+ ret = venus_iface_dbgq_read_nolock(hdev, pkt);
+ mutex_unlock(&hdev->lock);

+
+ return ret;
+}
+

+static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr)
+{
+ qhdr->status = 1;
+ qhdr->type = IFACEQ_DFLT_QHDR;
+ qhdr->q_size = IFACEQ_QUEUE_SIZE / 4;
+ qhdr->pkt_size = 0;
+ qhdr->rx_wm = 1;
+ qhdr->tx_wm = 1;
+ qhdr->rx_req = 1;
+ qhdr->tx_req = 0;
+ qhdr->rx_irq_status = 0;
+ qhdr->tx_irq_status = 0;
+ qhdr->read_idx = 0;
+ qhdr->write_idx = 0;
+}
+
+static void venus_interface_queues_release(struct venus_hfi_device *hdev)
+{
+ mutex_lock(&hdev->lock);
+
+ venus_free(hdev, &hdev->ifaceq_table);
+ venus_free(hdev, &hdev->sfr);
+
+ memset(hdev->queues, 0, sizeof(hdev->queues));
+ memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table));
+ memset(&hdev->sfr, 0, sizeof(hdev->sfr));
+
+ mutex_unlock(&hdev->lock);
+}
+
+static int venus_interface_queues_init(struct venus_hfi_device *hdev)
+{
+ struct hfi_queue_table_header *tbl_hdr;
+ struct iface_queue *queue;
+ struct hfi_sfr *sfr;
+ struct mem_desc desc = {0};
+ unsigned int offset;
+ unsigned int i;
+ int ret;
+
+ ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE);

+ if (ret)
+ return ret;
+

+ hdev->ifaceq_table.kva = desc.kva;
+ hdev->ifaceq_table.da = desc.da;
+ hdev->ifaceq_table.size = IFACEQ_TABLE_SIZE;
+ offset = hdev->ifaceq_table.size;
+
+ for (i = 0; i < IFACEQ_NUM; i++) {
+ queue = &hdev->queues[i];
+ queue->qmem.da = desc.da + offset;
+ queue->qmem.kva = desc.kva + offset;
+ queue->qmem.size = IFACEQ_QUEUE_SIZE;
+ offset += queue->qmem.size;
+ queue->qhdr =
+ IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i);
+
+ venus_set_qhdr_defaults(queue->qhdr);
+
+ queue->qhdr->start_addr = queue->qmem.da;
+
+ if (i == IFACEQ_CMD_IDX)
+ queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q;
+ else if (i == IFACEQ_MSG_IDX)
+ queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q;
+ else if (i == IFACEQ_DBG_IDX)
+ queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q;
+ }
+
+ tbl_hdr = hdev->ifaceq_table.kva;
+ tbl_hdr->version = 0;
+ tbl_hdr->size = IFACEQ_TABLE_SIZE;
+ tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header);
+ tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header);
+ tbl_hdr->num_q = IFACEQ_NUM;
+ tbl_hdr->num_active_q = IFACEQ_NUM;
+
+ /*
+ * Set receive request to zero on debug queue as there is no
+ * need of interrupt from video hardware for debug messages
+ */
+ queue = &hdev->queues[IFACEQ_DBG_IDX];
+ queue->qhdr->rx_req = 0;
+
+ ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE);
+ if (ret) {
+ hdev->sfr.da = 0;
+ } else {
+ hdev->sfr.da = desc.da;
+ hdev->sfr.kva = desc.kva;
+ hdev->sfr.size = ALIGNED_SFR_SIZE;
+ sfr = hdev->sfr.kva;
+ sfr->buf_size = ALIGNED_SFR_SIZE;
+ }
+
+ /* ensure table and queue header structs are settled in memory */
+ wmb();

+
+ return 0;
+}
+

+static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug)
+{
+ struct hfi_sys_set_property_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+ pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug);
+
+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode)
+{
+ struct hfi_sys_set_property_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+ pkt_sys_coverage_config(pkt, mode);
+
+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_sys_set_idle_message(struct venus_hfi_device *hdev,
+ bool enable)
+{
+ struct hfi_sys_set_property_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ if (!enable)
+ return 0;
+
+ pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+ pkt_sys_idle_indicator(pkt, enable);
+
+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_sys_set_power_control(struct venus_hfi_device *hdev,
+ bool enable)
+{
+ struct hfi_sys_set_property_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_sys_set_property_pkt *) packet;
+
+ pkt_sys_power_control(pkt, enable);
+
+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_get_queue_size(struct venus_hfi_device *hdev,
+ unsigned int index)
+{
+ struct hfi_queue_header *qhdr;
+
+ if (index >= IFACEQ_NUM)
+ return -EINVAL;
+
+ qhdr = hdev->queues[index].qhdr;
+ if (!qhdr)
+ return -EINVAL;
+
+ return abs(qhdr->read_idx - qhdr->write_idx);
+}
+
+static int venus_sys_set_default_properties(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ int ret;
+
+ ret = venus_sys_set_debug(hdev, venus_fw_debug);
+ if (ret)
+ dev_warn(dev, "setting fw debug msg ON failed (%d)\n", ret);
+
+ ret = venus_sys_set_idle_message(hdev, venus_sys_idle_indicator);
+ if (ret)
+ dev_warn(dev, "setting idle response ON failed (%d)\n", ret);
+
+ ret = venus_sys_set_power_control(hdev, venus_fw_low_power_mode);
+ if (ret)
+ dev_warn(dev, "setting hw power collapse ON failed (%d)\n",
+ ret);

+
+ return ret;
+}
+

+static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_pkt pkt;
+
+ pkt_session_cmd(&pkt, pkt_type, inst);
+
+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ void *packet = hdev->dbg_buf;
+
+ while (!venus_iface_dbgq_read(hdev, packet)) {
+ struct hfi_msg_sys_coverage_pkt *pkt = packet;
+
+ if (pkt->hdr.pkt_type != HFI_MSG_SYS_COV) {
+ struct hfi_msg_sys_debug_pkt *pkt = packet;
+
+ dev_dbg(dev, "%s", pkt->msg_data);
+ }
+ }
+}
+
+static int venus_prepare_power_collapse(struct venus_hfi_device *hdev,
+ bool wait)
+{
+ unsigned long timeout = msecs_to_jiffies(venus_hw_rsp_timeout);
+ struct hfi_sys_pc_prep_pkt pkt;
+ int ret;
+
+ init_completion(&hdev->pwr_collapse_prep);
+
+ pkt_sys_pc_prep(&pkt);
+
+ ret = venus_iface_cmdq_write(hdev, &pkt);

+ if (ret)
+ return ret;
+

+ if (!wait)
+ return 0;
+
+ ret = wait_for_completion_timeout(&hdev->pwr_collapse_prep, timeout);
+ if (!ret) {
+ venus_flush_debug_queue(hdev);
+ return -ETIMEDOUT;

+ }
+
+ return 0;
+}
+

+static int venus_are_queues_empty(struct venus_hfi_device *hdev)
+{
+ int ret1, ret2;
+
+ ret1 = venus_get_queue_size(hdev, IFACEQ_MSG_IDX);
+ if (ret1 < 0)
+ return ret1;
+
+ ret2 = venus_get_queue_size(hdev, IFACEQ_CMD_IDX);
+ if (ret2 < 0)
+ return ret2;
+
+ if (!ret1 && !ret2)
+ return 1;

+
+ return 0;
+}
+

+static void venus_sfr_print(struct venus_hfi_device *hdev)
+{
+ struct device *dev = hdev->core->dev;
+ struct hfi_sfr *sfr = hdev->sfr.kva;
+ void *p;
+
+ if (!sfr)
+ return;
+
+ p = memchr(sfr->data, '\0', sfr->buf_size);
+ /*
+ * SFR isn't guaranteed to be NULL terminated since SYS_ERROR indicates
+ * that Venus is in the process of crashing.
+ */
+ if (p == NULL)
+ sfr->data[sfr->buf_size - 1] = '\0';
+
+ dev_dbg_ratelimited(dev, "SFR message from FW: %s\n", sfr->data);
+}
+
+static void venus_process_msg_sys_error(struct venus_hfi_device *hdev,
+ void *packet)
+{
+ struct hfi_msg_event_notify_pkt *event_pkt = packet;
+
+ if (event_pkt->event_id != HFI_EVENT_SYS_ERROR)
+ return;
+
+ venus_set_state(hdev, VENUS_STATE_DEINIT);
+
+ /*
+ * Once SYS_ERROR received from HW, it is safe to halt the AXI.
+ * With SYS_ERROR, Venus FW may have crashed and HW might be
+ * active and causing unnecessary transactions. Hence it is
+ * safe to stop all AXI transactions from venus subsystem.
+ */
+ venus_halt_axi(hdev);
+ venus_sfr_print(hdev);
+}
+
+static irqreturn_t venus_isr_thread(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ const struct venus_resources *res = hdev->core->res;
+ void *pkt;
+ u32 msg_ret;
+
+ if (!hdev)
+ return IRQ_NONE;
+
+ pkt = hdev->pkt_buf;
+
+ if (hdev->irq_status & WRAPPER_INTR_STATUS_A2HWD_MASK) {
+ venus_sfr_print(hdev);
+ hfi_process_watchdog_timeout(core);
+ }
+
+ while (!venus_iface_msgq_read(hdev, pkt)) {
+ msg_ret = hfi_process_msg_packet(core, pkt);
+ switch (msg_ret) {
+ case HFI_MSG_EVENT_NOTIFY:
+ venus_process_msg_sys_error(hdev, pkt);
+ break;
+ case HFI_MSG_SYS_INIT:
+ venus_hfi_core_set_resource(core, res->vmem_id,
+ res->vmem_size,
+ res->vmem_addr,
+ hdev);
+ break;
+ case HFI_MSG_SYS_RELEASE_RESOURCE:
+ complete(&hdev->release_resource);
+ break;
+ case HFI_MSG_SYS_PC_PREP:
+ complete(&hdev->pwr_collapse_prep);
+ break;
+ default:
+ break;
+ }
+ }
+
+ venus_flush_debug_queue(hdev);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t venus_isr(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ u32 status;
+
+ if (!hdev)
+ return IRQ_NONE;
+
+ status = venus_readl(hdev, WRAPPER_INTR_STATUS);
+
+ if (status & WRAPPER_INTR_STATUS_A2H_MASK ||
+ status & WRAPPER_INTR_STATUS_A2HWD_MASK ||
+ status & CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK)
+ hdev->irq_status = status;
+
+ venus_writel(hdev, CPU_CS_A2HSOFTINTCLR, 1);
+ venus_writel(hdev, WRAPPER_INTR_CLEAR, status);
+
+ return IRQ_WAKE_THREAD;
+}
+
+static int venus_hfi_core_init(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct device *dev = core->dev;
+ struct hfi_sys_get_property_pkt version_pkt;
+ struct hfi_sys_init_pkt pkt;
+ int ret;
+
+ pkt_sys_init(&pkt, HFI_VIDEO_ARCH_OX);
+
+ venus_set_state(hdev, VENUS_STATE_INIT);
+
+ ret = venus_iface_cmdq_write(hdev, &pkt);

+ if (ret)
+ return ret;
+

+ pkt_sys_image_version(&version_pkt);
+
+ ret = venus_iface_cmdq_write(hdev, &version_pkt);
+ if (ret)
+ dev_warn(dev, "failed to send image version pkt to fw\n");

+
+ return 0;
+}
+

+static int venus_hfi_core_deinit(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+
+ venus_set_state(hdev, VENUS_STATE_DEINIT);
+ hdev->suspended = true;
+ hdev->power_enabled = false;

+
+ return 0;
+}
+

+static int venus_hfi_core_ping(struct venus_core *core, u32 cookie)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct hfi_sys_ping_pkt pkt;
+
+ pkt_sys_ping(&pkt, cookie);
+
+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_core_trigger_ssr(struct venus_core *core, u32 trigger_type)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct hfi_sys_test_ssr_pkt pkt;
+ int ret;
+
+ ret = pkt_sys_ssr_cmd(&pkt, trigger_type);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_session_init(struct venus_inst *inst,
+ u32 session_type, u32 codec)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_init_pkt pkt;
+ int ret;
+
+ ret = venus_sys_set_default_properties(hdev);

+ if (ret)
+ return ret;
+

+ ret = pkt_session_init(&pkt, inst, session_type, codec);
+ if (ret)
+ goto err;
+
+ ret = venus_iface_cmdq_write(hdev, &pkt);
+ if (ret)
+ goto err;

+
+ return 0;
+

+err:
+ venus_flush_debug_queue(hdev);

+ return ret;
+}
+

+static int venus_hfi_session_end(struct venus_inst *inst)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct device *dev = hdev->core->dev;
+
+ if (venus_fw_coverage) {
+ if (venus_sys_set_coverage(hdev, venus_fw_coverage))
+ dev_warn(dev, "fw coverage msg ON failed\n");
+ }
+
+ return venus_session_cmd(inst, HFI_CMD_SYS_SESSION_END);
+}
+
+static int venus_hfi_session_abort(struct venus_inst *inst)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+
+ venus_flush_debug_queue(hdev);
+
+ return venus_session_cmd(inst, HFI_CMD_SYS_SESSION_ABORT);
+}
+
+static int venus_hfi_session_flush(struct venus_inst *inst, u32 flush_mode)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_flush_pkt pkt;
+ int ret;
+
+ ret = pkt_session_flush(&pkt, inst, flush_mode);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_session_start(struct venus_inst *inst)
+{
+ return venus_session_cmd(inst, HFI_CMD_SESSION_START);
+}
+
+static int venus_hfi_session_stop(struct venus_inst *inst)
+{
+ return venus_session_cmd(inst, HFI_CMD_SESSION_STOP);
+}
+
+static int venus_hfi_session_continue(struct venus_inst *inst)
+{
+ return venus_session_cmd(inst, HFI_CMD_SESSION_CONTINUE);
+}
+
+static int venus_hfi_session_etb(struct venus_inst *inst,
+ struct hfi_frame_data *in_frame)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ u32 session_type = inst->session_type;
+ int ret;
+
+ if (session_type == VIDC_SESSION_TYPE_DEC) {
+ struct hfi_session_empty_buffer_compressed_pkt pkt;
+
+ ret = pkt_session_etb_decoder(&pkt, inst, in_frame);

+ if (ret)
+ return ret;
+

+ ret = venus_iface_cmdq_write(hdev, &pkt);
+ } else if (session_type == VIDC_SESSION_TYPE_ENC) {
+ struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt;
+
+ ret = pkt_session_etb_encoder(&pkt, inst, in_frame);

+ if (ret)
+ return ret;
+

+ ret = venus_iface_cmdq_write(hdev, &pkt);
+ } else {

+ ret = -EINVAL;
+ }

+
+ return ret;
+}
+

+static int venus_hfi_session_ftb(struct venus_inst *inst,
+ struct hfi_frame_data *out_frame)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_fill_buffer_pkt pkt;
+ int ret;
+
+ ret = pkt_session_ftb(&pkt, inst, out_frame);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_session_set_buffers(struct venus_inst *inst,
+ struct hfi_buffer_desc *bd)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_set_buffers_pkt *pkt;
+ u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+ int ret;
+
+ if (bd->buffer_type == HFI_BUFFER_INPUT)
+ return 0;
+
+ pkt = (struct hfi_session_set_buffers_pkt *)packet;
+
+ ret = pkt_session_set_buffers(pkt, inst, bd);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_unset_buffers(struct venus_inst *inst,
+ struct hfi_buffer_desc *bd)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_release_buffer_pkt *pkt;
+ u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+ int ret;
+
+ if (bd->buffer_type == HFI_BUFFER_INPUT)
+ return 0;
+
+ pkt = (struct hfi_session_release_buffer_pkt *) packet;
+
+ ret = pkt_session_unset_buffers(pkt, inst, bd);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_load_res(struct venus_inst *inst)
+{
+ return venus_session_cmd(inst, HFI_CMD_SESSION_LOAD_RESOURCES);
+}
+
+static int venus_hfi_session_release_res(struct venus_inst *inst)
+{
+ return venus_session_cmd(inst, HFI_CMD_SESSION_RELEASE_RESOURCES);
+}
+
+static int venus_hfi_session_parse_seq_hdr(struct venus_inst *inst,
+ u32 seq_hdr, u32 seq_hdr_len)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_parse_sequence_header_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_session_parse_sequence_header_pkt *) packet;
+
+ ret = pkt_session_parse_seq_header(pkt, inst, seq_hdr, seq_hdr_len);

+ if (ret)
+ return ret;
+

+ ret = venus_iface_cmdq_write(hdev, pkt);

+ if (ret)
+ return ret;
+
+ return 0;
+}
+

+static int venus_hfi_session_get_seq_hdr(struct venus_inst *inst,
+ u32 seq_hdr, u32 seq_hdr_len)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_get_sequence_header_pkt *pkt;
+ u8 packet[IFACEQ_VAR_SMALL_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_session_get_sequence_header_pkt *) packet;
+
+ ret = pkt_session_get_seq_hdr(pkt, inst, seq_hdr, seq_hdr_len);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_set_property(struct venus_inst *inst, u32 ptype,
+ void *pdata)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_set_property_pkt *pkt;
+ u8 packet[IFACEQ_VAR_LARGE_PKT_SIZE];
+ int ret;
+
+ pkt = (struct hfi_session_set_property_pkt *) packet;
+
+ ret = pkt_session_set_property(pkt, inst, ptype, pdata);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, pkt);
+}
+
+static int venus_hfi_session_get_property(struct venus_inst *inst, u32 ptype)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
+ struct hfi_session_get_property_pkt pkt;
+ int ret;
+
+ ret = pkt_session_get_property(&pkt, inst, ptype);

+ if (ret)
+ return ret;
+

+ return venus_iface_cmdq_write(hdev, &pkt);
+}
+
+static int venus_hfi_resume(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);

+ int ret = 0;
+

+ mutex_lock(&hdev->lock);
+
+ if (hdev->suspended == false)
+ goto unlock;
+
+ ret = venus_power_on(hdev);
+
+unlock:
+ if (!ret)
+ hdev->suspended = false;
+
+ mutex_unlock(&hdev->lock);

+
+ return ret;
+}
+

+static int venus_hfi_suspend_1xx(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct device *dev = core->dev;
+ u32 ctrl_status;
+ int ret;
+
+ if (!hdev->power_enabled || hdev->suspended)
+ return 0;
+
+ mutex_lock(&hdev->lock);
+ ret = venus_is_valid_state(hdev);
+ mutex_unlock(&hdev->lock);
+
+ if (!ret) {
+ dev_err(dev, "bad state, cannot suspend\n");

+ return -EINVAL;
+ }
+

+ ret = venus_prepare_power_collapse(hdev, true);
+ if (ret) {
+ dev_err(dev, "prepare for power collapse fail (%d)\n", ret);

+ return ret;
+ }
+

+ mutex_lock(&hdev->lock);
+
+ if (hdev->last_packet_type != HFI_CMD_SYS_PC_PREP) {
+ mutex_unlock(&hdev->lock);

+ return -EINVAL;
+ }
+

+ ret = venus_are_queues_empty(hdev);
+ if (ret < 0 || !ret) {
+ mutex_unlock(&hdev->lock);

+ return -EINVAL;
+ }
+

+ ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+ if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
+ mutex_unlock(&hdev->lock);

+ return -EINVAL;
+ }
+

+ ret = venus_power_off(hdev);
+ if (ret) {
+ mutex_unlock(&hdev->lock);

+ return ret;
+ }
+

+ hdev->suspended = true;
+
+ mutex_unlock(&hdev->lock);

+
+ return 0;
+}
+

+static int venus_hfi_suspend_3xx(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+ struct device *dev = core->dev;
+ u32 ctrl_status, wfi_status;
+ int ret;
+ int cnt = 100;
+
+ if (!hdev->power_enabled || hdev->suspended)
+ return 0;
+
+ mutex_lock(&hdev->lock);
+ ret = venus_is_valid_state(hdev);
+ mutex_unlock(&hdev->lock);
+
+ if (!ret) {
+ dev_err(dev, "bad state, cannot suspend\n");

+ return -EINVAL;
+ }
+

+ ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+ if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
+ wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
+ ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+
+ ret = venus_prepare_power_collapse(hdev, false);
+ if (ret) {
+ dev_err(dev, "prepare for power collapse fail (%d)\n",
+ ret);

+ return ret;
+ }
+

+ cnt = 100;
+ while (cnt--) {
+ wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
+ ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+ if (ctrl_status & CPU_CS_SCIACMDARG0_PC_READY &&
+ wfi_status & BIT(0))
+ break;
+ usleep_range(1000, 1500);
+ }
+ }
+
+ mutex_lock(&hdev->lock);
+
+ ret = venus_power_off(hdev);
+ if (ret) {
+ dev_err(dev, "venus_power_off (%d)\n", ret);
+ mutex_unlock(&hdev->lock);

+ return ret;
+ }
+

+ hdev->suspended = true;
+
+ mutex_unlock(&hdev->lock);

+
+ return 0;
+}
+

+static int venus_hfi_suspend(struct venus_core *core)

+{
+ if (core->res->hfi_version == HFI_VERSION_3XX)

+ return venus_hfi_suspend_3xx(core);
+
+ return venus_hfi_suspend_1xx(core);
+}
+
+static const struct hfi_ops venus_hfi_ops = {
+ .core_init = venus_hfi_core_init,
+ .core_deinit = venus_hfi_core_deinit,
+ .core_ping = venus_hfi_core_ping,
+ .core_trigger_ssr = venus_hfi_core_trigger_ssr,
+
+ .session_init = venus_hfi_session_init,
+ .session_end = venus_hfi_session_end,
+ .session_abort = venus_hfi_session_abort,
+ .session_flush = venus_hfi_session_flush,
+ .session_start = venus_hfi_session_start,
+ .session_stop = venus_hfi_session_stop,
+ .session_continue = venus_hfi_session_continue,
+ .session_etb = venus_hfi_session_etb,
+ .session_ftb = venus_hfi_session_ftb,
+ .session_set_buffers = venus_hfi_session_set_buffers,
+ .session_unset_buffers = venus_hfi_session_unset_buffers,
+ .session_load_res = venus_hfi_session_load_res,
+ .session_release_res = venus_hfi_session_release_res,
+ .session_parse_seq_hdr = venus_hfi_session_parse_seq_hdr,
+ .session_get_seq_hdr = venus_hfi_session_get_seq_hdr,
+ .session_set_property = venus_hfi_session_set_property,
+ .session_get_property = venus_hfi_session_get_property,
+
+ .resume = venus_hfi_resume,
+ .suspend = venus_hfi_suspend,
+
+ .isr = venus_isr,
+ .isr_thread = venus_isr_thread,
+};
+
+void venus_hfi_destroy(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev = to_hfi_priv(core);
+
+ venus_interface_queues_release(hdev);
+ mutex_destroy(&hdev->lock);
+ kfree(hdev);
+ core->priv = NULL;
+ core->ops = NULL;
+}
+
+int venus_hfi_create(struct venus_core *core)
+{
+ struct venus_hfi_device *hdev;
+ int ret;
+
+ hdev = kzalloc(sizeof(*hdev), GFP_KERNEL);
+ if (!hdev)
+ return -ENOMEM;
+
+ mutex_init(&hdev->lock);
+
+ hdev->core = core;
+ hdev->suspended = true;
+ core->priv = hdev;
+ core->ops = &venus_hfi_ops;
+ core->core_caps = ENC_ROTATION_CAPABILITY | ENC_SCALING_CAPABILITY |
+ ENC_DEINTERLACE_CAPABILITY |
+ DEC_MULTI_STREAM_CAPABILITY;
+
+ ret = venus_interface_queues_init(hdev);
+ if (ret)
+ goto err_kfree;

+
+ return 0;
+

+err_kfree:
+ kfree(hdev);
+ core->priv = NULL;
+ core->ops = NULL;
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/venus/hfi_venus.h
new file mode 100644
index 000000000000..885923354033
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_venus.h

@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */

+#ifndef __VENUS_HFI_VENUS_H__
+#define __VENUS_HFI_VENUS_H__
+
+struct venus_core;
+
+void venus_hfi_destroy(struct venus_core *core);
+int venus_hfi_create(struct venus_core *core);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/venus/hfi_venus_io.h
new file mode 100644
index 000000000000..98cc350113ab
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_venus_io.h
@@ -0,0 +1,113 @@

+/*
+ * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2017 Linaro Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */

+#ifndef __VENUS_HFI_VENUS_IO_H__
+#define __VENUS_HFI_VENUS_IO_H__
+
+#define VBIF_BASE 0x80000
+
+#define VBIF_AXI_HALT_CTRL0 (VBIF_BASE + 0x208)
+#define VBIF_AXI_HALT_CTRL1 (VBIF_BASE + 0x20c)
+
+#define VBIF_AXI_HALT_CTRL0_HALT_REQ BIT(0)
+#define VBIF_AXI_HALT_CTRL1_HALT_ACK BIT(0)
+#define VBIF_AXI_HALT_ACK_TIMEOUT_US 500000
+
+#define CPU_BASE 0xc0000
+#define CPU_CS_BASE (CPU_BASE + 0x12000)
+#define CPU_IC_BASE (CPU_BASE + 0x1f000)
+
+#define CPU_CS_A2HSOFTINTCLR (CPU_CS_BASE + 0x1c)
+
+#define VIDC_CTRL_INIT (CPU_CS_BASE + 0x48)
+#define VIDC_CTRL_INIT_RESERVED_BITS31_1_MASK 0xfffffffe
+#define VIDC_CTRL_INIT_RESERVED_BITS31_1_SHIFT 1
+#define VIDC_CTRL_INIT_CTRL_MASK 0x1
+#define VIDC_CTRL_INIT_CTRL_SHIFT 0
+
+/* HFI control status */
+#define CPU_CS_SCIACMDARG0 (CPU_CS_BASE + 0x4c)
+#define CPU_CS_SCIACMDARG0_MASK 0xff
+#define CPU_CS_SCIACMDARG0_SHIFT 0x0
+#define CPU_CS_SCIACMDARG0_ERROR_STATUS_MASK 0xfe
+#define CPU_CS_SCIACMDARG0_ERROR_STATUS_SHIFT 0x1
+#define CPU_CS_SCIACMDARG0_INIT_STATUS_MASK 0x1
+#define CPU_CS_SCIACMDARG0_INIT_STATUS_SHIFT 0x0
+#define CPU_CS_SCIACMDARG0_PC_READY BIT(8)
+#define CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK BIT(30)
+
+/* HFI queue table info */
+#define CPU_CS_SCIACMDARG1 (CPU_CS_BASE + 0x50)
+
+/* HFI queue table address */
+#define CPU_CS_SCIACMDARG2 (CPU_CS_BASE + 0x54)
+
+/* Venus cpu */
+#define CPU_CS_SCIACMDARG3 (CPU_CS_BASE + 0x58)
+
+#define SFR_ADDR (CPU_CS_BASE + 0x5c)
+#define MMAP_ADDR (CPU_CS_BASE + 0x60)
+#define UC_REGION_ADDR (CPU_CS_BASE + 0x64)
+#define UC_REGION_SIZE (CPU_CS_BASE + 0x68)
+
+#define CPU_IC_SOFTINT (CPU_IC_BASE + 0x18)
+#define CPU_IC_SOFTINT_H2A_MASK 0x8000
+#define CPU_IC_SOFTINT_H2A_SHIFT 0xf
+
+/* Venus wrapper */
+#define WRAPPER_BASE 0x000e0000
+
+#define WRAPPER_HW_VERSION (WRAPPER_BASE + 0x00)
+#define WRAPPER_HW_VERSION_MAJOR_VERSION_MASK 0x78000000
+#define WRAPPER_HW_VERSION_MAJOR_VERSION_SHIFT 28
+#define WRAPPER_HW_VERSION_MINOR_VERSION_MASK 0xfff0000
+#define WRAPPER_HW_VERSION_MINOR_VERSION_SHIFT 16
+#define WRAPPER_HW_VERSION_STEP_VERSION_MASK 0xffff
+
+#define WRAPPER_CLOCK_CONFIG (WRAPPER_BASE + 0x04)
+
+#define WRAPPER_INTR_STATUS (WRAPPER_BASE + 0x0c)
+#define WRAPPER_INTR_STATUS_A2HWD_MASK 0x10
+#define WRAPPER_INTR_STATUS_A2HWD_SHIFT 0x4
+#define WRAPPER_INTR_STATUS_A2H_MASK 0x4
+#define WRAPPER_INTR_STATUS_A2H_SHIFT 0x2
+
+#define WRAPPER_INTR_MASK (WRAPPER_BASE + 0x10)
+#define WRAPPER_INTR_MASK_A2HWD_BASK 0x10
+#define WRAPPER_INTR_MASK_A2HWD_SHIFT 0x4
+#define WRAPPER_INTR_MASK_A2HVCODEC_MASK 0x8
+#define WRAPPER_INTR_MASK_A2HVCODEC_SHIFT 0x3
+#define WRAPPER_INTR_MASK_A2HCPU_MASK 0x4
+#define WRAPPER_INTR_MASK_A2HCPU_SHIFT 0x2
+
+#define WRAPPER_INTR_CLEAR (WRAPPER_BASE + 0x14)
+#define WRAPPER_INTR_CLEAR_A2HWD_MASK 0x10
+#define WRAPPER_INTR_CLEAR_A2HWD_SHIFT 0x4
+#define WRAPPER_INTR_CLEAR_A2H_MASK 0x4
+#define WRAPPER_INTR_CLEAR_A2H_SHIFT 0x2
+
+#define WRAPPER_POWER_STATUS (WRAPPER_BASE + 0x44)
+#define WRAPPER_VDEC_VCODEC_POWER_CONTROL (WRAPPER_BASE + 0x48)
+#define WRAPPER_VENC_VCODEC_POWER_CONTROL (WRAPPER_BASE + 0x4c)
+#define WRAPPER_VDEC_VENC_AHB_BRIDGE_SYNC_RESET (WRAPPER_BASE + 0x64)
+
+#define WRAPPER_CPU_CLOCK_CONFIG (WRAPPER_BASE + 0x2000)
+#define WRAPPER_CPU_AXI_HALT (WRAPPER_BASE + 0x2008)
+#define WRAPPER_CPU_AXI_HALT_STATUS (WRAPPER_BASE + 0x200c)
+
+#define WRAPPER_CPU_CGC_DIS (WRAPPER_BASE + 0x2010)
+#define WRAPPER_CPU_STATUS (WRAPPER_BASE + 0x2014)
+#define WRAPPER_SW_RESET (WRAPPER_BASE + 0x3000)
+
+#endif
--
2.7.4

[Hans Verkuil]

On 03/24/17 15:06, Hans Verkuil wrote:

> On 03/13/17 17:37, Stanimir Varbanov wrote:
>> Hi all,
>>
>> Here is seventh version of the patch-set - no functional changes in
>> v4l2 APIs.
>>
>> The changes since v6 are.
>> * changes in DT binding document - moved memory-region DT property
>> in video-codec node - see 2/9.
>> * improved recovery mechanism.
>> * fixed various issues found during testing.
>>
>> Build dependencies:
>> - qcom_scm_set_remote_state is merged in Linux 4.11-rc1
>> - qcom mdt_loader is merged in Linux 4.11-rc1
>>
>> regards,
>> Stan
>>
>> Stanimir Varbanov (9):
>> media: v4l2-mem2mem: extend m2m APIs for more accurate buffer
>> management
>

> I don't see this patch on linux-media. Can you post that one again?

Urgh, never mind. It's there, my mistake.

>
> Can you also post the v4l2-compliance test results? Make sure you compile
> v4l2-compliance straight from the git repo to avoid testing with an old
> version.

This I'd still like to see.

Hans

>
> Thanks!
>
> Hans

[Hans Verkuil]

On 03/13/17 17:37, Stanimir Varbanov wrote:

> Hi all,
>
> Here is seventh version of the patch-set - no functional changes in
> v4l2 APIs.
>
> The changes since v6 are.
> * changes in DT binding document - moved memory-region DT property
> in video-codec node - see 2/9.
> * improved recovery mechanism.
> * fixed various issues found during testing.
>
> Build dependencies:
> - qcom_scm_set_remote_state is merged in Linux 4.11-rc1
> - qcom mdt_loader is merged in Linux 4.11-rc1
>
> regards,
> Stan
>
> Stanimir Varbanov (9):
> media: v4l2-mem2mem: extend m2m APIs for more accurate buffer
> management

I don't see this patch on linux-media. Can you post that one again?

Can you also post the v4l2-compliance test results? Make sure you compile
v4l2-compliance straight from the git repo to avoid testing with an old
version.

Thanks!

Hans

[Hans Verkuil]

Some comments and questions below:

On 03/13/17 17:37, Stanimir Varbanov wrote:

Just curious: is NV12 the only uncompressed format supported by the hardware?
Or just the only one that is implemented here?

num_planes is always 1, do you need it at all? And if it is always one,
why use _MPLANE at all? Is this for future additions?

timestamp and sequence are only set for CAPTURE, not OUTPUT. Is that correct?

Regards,

Hans

[Hans Verkuil]

On 03/13/17 17:37, Stanimir Varbanov wrote:

> This adds Venus driver Makefile and changes v4l2 platform
> Makefile/Kconfig in order to enable building of the driver.
>
> Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
> ---
> drivers/media/platform/Kconfig | 14 ++++++++++++++
> drivers/media/platform/Makefile | 2 ++
> drivers/media/platform/qcom/venus/Makefile | 11 +++++++++++
> 3 files changed, 27 insertions(+)
> create mode 100644 drivers/media/platform/qcom/venus/Makefile
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 53f6f12bff0d..8a6c3d664307 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -447,6 +447,20 @@ config VIDEO_TI_VPE_DEBUG
> ---help---
> Enable debug messages on VPE driver.
>
> +config VIDEO_QCOM_VENUS
> + tristate "Qualcomm Venus V4L2 encoder/decoder driver"
> + depends on VIDEO_DEV && VIDEO_V4L2 && HAS_DMA

Can this also depend on COMPILE_TEST? And if so, please make sure it compile on both
a 32 and 64 bit environment to shake out any compiler warnings.

Regards,

Hans

[Stanimir Varbanov]

Thanks for the comments!

On 03/24/2017 04:49 PM, Hans Verkuil wrote:
> On 03/13/17 17:37, Stanimir Varbanov wrote:
>> This adds Venus driver Makefile and changes v4l2 platform
>> Makefile/Kconfig in order to enable building of the driver.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir...@linaro.org>
>> ---
>> drivers/media/platform/Kconfig | 14 ++++++++++++++
>> drivers/media/platform/Makefile | 2 ++
>> drivers/media/platform/qcom/venus/Makefile | 11 +++++++++++
>> 3 files changed, 27 insertions(+)
>> create mode 100644 drivers/media/platform/qcom/venus/Makefile
>>
>> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
>> index 53f6f12bff0d..8a6c3d664307 100644
>> --- a/drivers/media/platform/Kconfig
>> +++ b/drivers/media/platform/Kconfig
>> @@ -447,6 +447,20 @@ config VIDEO_TI_VPE_DEBUG
>> ---help---
>> Enable debug messages on VPE driver.
>>
>> +config VIDEO_QCOM_VENUS
>> + tristate "Qualcomm Venus V4L2 encoder/decoder driver"
>> + depends on VIDEO_DEV && VIDEO_V4L2 && HAS_DMA
>
> Can this also depend on COMPILE_TEST? And if so, please make sure it compile on both
> a 32 and 64 bit environment to shake out any compiler warnings.
>

yes I can add COMPILE_TEST, at least on -next it should be fine.

--
regards,
Stan

[Nicolas Dufresne]

Le vendredi 24 mars 2017 à 15:41 +0100, Hans Verkuil a écrit :
> > +static const struct venus_format vdec_formats[] = {
> > +     {
> > +             .pixfmt = V4L2_PIX_FMT_NV12,
> > +             .num_planes = 1,
> > +             .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>
> Just curious: is NV12 the only uncompressed format supported by the
> hardware?
> Or just the only one that is implemented here?

The downstream kernel[0], from Qualcomm have:

{
                .name = "UBWC YCbCr Semiplanar 4:2:0",
                .description = "UBWC Y/CbCr 4:2:0",
                .fourcc = V4L2_PIX_FMT_NV12_UBWC,
                .num_planes = 2,
                .get_frame_size = get_frame_size_nv12_ubwc,
                .type = CAPTURE_PORT,
        },
        {
                .name = "UBWC YCbCr Semiplanar 4:2:0 10bit",
                .description = "UBWC Y/CbCr 4:2:0 10bit",
                .fourcc = V4L2_PIX_FMT_NV12_TP10_UBWC,
                .num_planes = 2,
                .get_frame_size = get_frame_size_nv12_ubwc_10bit,
                .type = CAPTURE_PORT,
        },

I have no idea what UBWC stands for. The performance in NV12 is more
then decent from my testing. Though, there is no 10bit variant.

regards,
Nicolas

[0] https://android.googlesource.com/kernel/msm/+/android-7.1.0_r0.2/dr
ivers/media/platform/msm/vidc/msm_vdec.c#695

[Stanimir Varbanov]

Thanks for the comments!

<snip> three reasons:
- _MPLAIN allows one plane only
- downstream qualcomm driver use _MPLAIN (the second plain is used for
extaradata, I ignored the extaradata support for now until v4l2 metadata
api is merged)
- I still believe that qualcomm firmware guys will add support the
second or even third plain at some point.

Correct. I can add sequence for the OUTPUT queue too, but I have no idea
how that sequence is used by userspace.

>
>> +
>> + if (vbuf->flags & V4L2_BUF_FLAG_LAST) {
>> + const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
>> +
>> + v4l2_event_queue_fh(&inst->fh, &ev);
>> + }
>> + }
>> +
>> + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
>> +}

<snip>

--
regards,
Stan

[Stanimir Varbanov]

Hi,

On 24.03.2017 20:21, Nicolas Dufresne wrote:
> Le vendredi 24 mars 2017 à 15:41 +0100, Hans Verkuil a écrit :
>>> +static const struct venus_format vdec_formats[] = {
>>> + {
>>> + .pixfmt = V4L2_PIX_FMT_NV12,
>>> + .num_planes = 1,
>>> + .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>
>> Just curious: is NV12 the only uncompressed format supported by the
>> hardware?
>> Or just the only one that is implemented here?

yes, at least to my knowledge (except below UBWC).

>
> The downstream kernel[0], from Qualcomm have:
>
> {
> .name = "UBWC YCbCr Semiplanar 4:2:0",
> .description = "UBWC Y/CbCr 4:2:0",
> .fourcc = V4L2_PIX_FMT_NV12_UBWC,
> .num_planes = 2,
> .get_frame_size = get_frame_size_nv12_ubwc,
> .type = CAPTURE_PORT,
> },
> {
> .name = "UBWC YCbCr Semiplanar 4:2:0 10bit",
> .description = "UBWC Y/CbCr 4:2:0 10bit",
> .fourcc = V4L2_PIX_FMT_NV12_TP10_UBWC,
> .num_planes = 2,
> .get_frame_size = get_frame_size_nv12_ubwc_10bit,
> .type = CAPTURE_PORT,
> },
>
> I have no idea what UBWC stands for. The performance in NV12 is more
> then decent from my testing. Though, there is no 10bit variant.

UBWC is some kind of compressed format for NV12 [1]. This format is
applicable for the newer venus hardware revisions and I planed to add it
later on (when Adreno GPU driver starts handle it).

regards,
Stan

[1]
https://android.googlesource.com/kernel/msm/+/android-7.1.0_r0.2/include/media/msm_media_info.h#151

[Nicolas Dufresne]

Le dimanche 26 mars 2017 à 00:30 +0200, Stanimir Varbanov a écrit :
> > > +            vb->planes[0].data_offset = data_offset;
> > > +            vb->timestamp = timestamp_us * NSEC_PER_USEC;
> > > +            vbuf->sequence = inst->sequence++;
> >
> > timestamp and sequence are only set for CAPTURE, not OUTPUT. Is
> > that correct?
>
> Correct. I can add sequence for the OUTPUT queue too, but I have no idea 
> how that sequence is used by userspace.

Neither GStreamer or Chromium seems to use it. What does that number
means for a m2m driver ? Does it really means something ?

Nicolas

[Hans Verkuil]

Please add a comment in the code explaining the reason. Just before this format list would be
a good place for that.

Hans

[Hans Verkuil]

On 25/03/17 23:30, Stanimir Varbanov wrote:

You set V4L2_BUF_FLAG_TIMESTAMP_COPY, so you have to copy the timestamp from the output buffer
to the capture buffer, if that makes sense for this codec. If not, then you shouldn't use that
V4L2_BUF_FLAG and just generate new timestamps whenever a capture buffer is ready.

For sequence numbering just give the output queue its own sequence counter.

Regards,

Hans

[Hans Verkuil]

It can be used to detect dropped frame (the sequence counter will skip in that
case).

Unlikely to happen for m2m devices, and most apps ignore it as well. But you
still need to fill it in, it's a V4L2 requirement.

Regards,

Hans

[Stanimir Varbanov]

Hi Hans,

On 03/27/2017 11:45 AM, Hans Verkuil wrote:
> On 25/03/17 23:30, Stanimir Varbanov wrote:
>> Thanks for the comments!

<snip>

The timestamp_us is filled by firmware and it is the timestamp of the
output buffer which is used to produce the uncompressed capture buffer.
So I think V4L2_BUF_FLAG_TIMESTAMP_COPY is correctly used here.

> V4L2_BUF_FLAG and just generate new timestamps whenever a capture buffer is ready.
>
> For sequence numbering just give the output queue its own sequence counter.

OK will do.

--
regards,
Stan

[Nicolas Dufresne]

Le lundi 27 mars 2017 à 10:45 +0200, Hans Verkuil a écrit :
> > > timestamp and sequence are only set for CAPTURE, not OUTPUT. Is
> > > that correct?
> >
> > Correct. I can add sequence for the OUTPUT queue too, but I have no
> > idea how that sequence is used by userspace.
>
> You set V4L2_BUF_FLAG_TIMESTAMP_COPY, so you have to copy the
> timestamp from the output buffer
> to the capture buffer, if that makes sense for this codec. If not,
> then you shouldn't use that
> V4L2_BUF_FLAG and just generate new timestamps whenever a capture
> buffer is ready.
>
> For sequence numbering just give the output queue its own sequence
> counter.

Btw, GStreamer and Chromium only supports TIMESTAMP_COPY, and will most
likely leak frames if you craft timestamp.

Nicolas