Interfacing with veejay

45 views
Skip to first unread message

Patric Schmitz

unread,
Jan 27, 2016, 1:21:39 AM1/27/16
to veejay-d...@googlegroups.com, Claudio, Sevinc
Hi everyone,

I'm making progress on my opensoundcontrol'ed rendering project
and would like to slowly start on the integration with veejay.

What's the most reasonable way to provide live rendered output as
input to veejay, I suppose a v4l2 loopback device for now? This
will require copying back and forth between GPU and host I
assume? (Which is fine for the time being.)

As you probably know there is this Syphon thing for Mac which
allows video applications to share texture/image/video data
directly on the GPU side. Is there anything like this already in
veejay, or available in general on Linux, which one could make
use of? I read this interesting thread on the mapmap mailing list
some weeks ago, it seems like the next incarnation of DRI might
provide this kind of functionality through the DMA-buf mechanism:
> http://keithp.com/blogs/DRI-Next/
> https://github.com/pcwalton/linux-drm-sharing-test

My idea was that the different renderers I create each render
into offscreen buffers (OpenGL FBO) and present those as seperate
sources to veejay, so that application of effects as well as
compositing/layering can be done on the veejay side.

I would be very interested in any pointers on how to go about
this, reference/example implementations I could look into as well
as ideas on how to do this in the most efficient way, since for
the kind of live visual setup I'm planning latency is of course
kinda critical.

Thanks in advance!

Niels Elburg

unread,
Jan 27, 2016, 12:19:48 PM1/27/16
to veejay-discussion
Hi Patric,

Yes, using something with a  v4l2 loopback device would be the quickest setup
(like ffmpeg -f x11grab -r 25 -s 720x576 -i :0.0+0,0 -vcodec rawvideo -pix_fmt yuv422p -f v4l2 /dev/video0)
and it may even be fast enough, depending on the size of the source buffer

I am not aware of anything that resembles Syphon (on Linux)  ... or what the status is of "similar" projects

How to go about this, well that's a tough one!

I am not sure if FBO can be shared between processes, my openGL skills are a bit rusty (or outdated)

Perhaps you can create a PBO, bind the PBO as GL_PIXEL_PACK_BUFFER and do glReadPixels to transfer it to (veejay's) shared memory resource
The glReadPixels call was notorious for its horrible performance, but perhaps this has improved over the past few years?

If you have something working that renders something to an offscreen buffer, I could take a look and see if it can be copied to veejays shm

Either solution should be fast enough,

See you,
Niels




--
You received this message because you are subscribed to the Google Groups "veejay-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to veejay-discuss...@googlegroups.com.
To post to this group, send email to veejay-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/veejay-discussion.
For more options, visit https://groups.google.com/d/optout.

Patric Schmitz

unread,
Jan 27, 2016, 2:57:01 PM1/27/16
to veejay-d...@googlegroups.com, Claudio, Sevinc
Heyo,

On 01/27/2016 06:19 PM, Niels Elburg wrote:
> Yes, using something with a v4l2 loopback device would be the
> quickest setup
> (like ffmpeg -f x11grab -r 25 -s 720x576 -i :0.0+0,0 -vcodec
> rawvideo -pix_fmt yuv422p -f v4l2 /dev/video0)
> and it may even be fast enough, depending on the size of the
> source buffer

I was thinking about using the v4l API directly from within my
application, not going the roundtrip of first rendering to the
screen, grabbing from there and then sending to veejay via v4l.
This way I can also have several non-visible layers rendering and
streaming via v4l in parallel.

> I am not aware of anything that resembles Syphon (on Linux) ...
> or what the status is of "similar" projects

Ok will go with v4l for now then, might keep an eye on what's
happening with that DRI 3 development.

> How to go about this, well that's a tough one!
> ...

Getting the pixel data out of an FBO should not be an issue, I
was more asking about the v4l API side of things. I will just
look through the docs for now and see how far I get. Otherwise
the v4l2 output driver of ffmpeg will probably contain the code
I'm searching for.

> If you have something working that renders something to an
> offscreen buffer, I could take a look and see if it can be copied
> to veejays shm

Direct communication via shm might of course be an option,
however I'd look into that only after checking how it performs
when using the v4l API directly. Using this will keep it more
generic and I can e.g. interface with mapmap directly, or
restream the 'device' over a network with vlc etc. so it would be
more generically useful for the time being.

Thanks, I'll keep you updated on any progress!

Patric

Niels Elburg

unread,
Jan 27, 2016, 3:07:58 PM1/27/16
to veejay-discussion
Alright,

You could also take a look at vj-vloopback.c (https://github.com/c0ntrol/veejay/blob/master/veejay-current/veejay-server/libstream/vj-vloopback.c) which is veejay's v4l(2) loopback writer

Looking forward to your progress :)




Patric

Patric Schmitz

unread,
Jan 27, 2016, 3:23:28 PM1/27/16
to veejay-d...@googlegroups.com
On 01/27/2016 09:07 PM, Niels Elburg wrote:
> You could also take a look at vj-vloopback.c
> (https://github.com/c0ntrol/veejay/blob/master/veejay-current/veejay-server/libstream/vj-vloopback.c)
> which is veejay's v4l(2) loopback writer

Ah that's what I have been looking for, great!
Cool might look into it as well if v4l doesn't perform nicely
latency wise, but I suppose it will be just fine for now.

btw just added veejay to the list of software supporting v4l
while reading the wikipedia pages :)
> https://en.wikipedia.org/wiki/Video4Linux#Software_supporting_Video4Linux

Cheers!
Reply all
Reply to author
Forward
0 new messages