Acessing generic data in shaders

[Robert Oetting]

Hi everyone!

I want to make java array/vector data available in shaders as 2dtextures or tbo's (Mostly for accessing audio data like wave-forms, spectrograms etc. in fragment shaders).

Since im new to java and praxis-live architecture, im confused about how to do this. What graphics-api should i use for this? The processing api? Plane opengl-java bindings?

The most obvious way, as it seems to me, would be to write PImage-pixel data. But thats not possible as far as I can tell./!/? Also the texture should not be 8-bit-rgb, but float-grayscale. Texture buffer objects are preferred because of integer-indexing (but thats not really important)

I think it would be great if I could manage to extend praxis-live in a way that shader.set(string "name", float or double array data) generates/updates a tbo.

Or am I missing something obvious here?

best regards,

  robert oetting

[Neil C Smith]

On Mon, 9 Nov 2020 at 09:57, Robert Oetting
<oetting.ro...@gmail.com> wrote:
> Hi everyone!

Hi Robert, and welcome to the list!

> I want to make java array/vector data available in shaders as 2dtextures or tbo's (Mostly for accessing audio data like wave-forms, spectrograms etc. in fragment shaders).

Sounds interesting - keep meaning to have a look at doing this myself.

> Since im new to java and praxis-live architecture, im confused about how to do this. What graphics-api should i use for this? The processing api? Plane opengl-java bindings?

Well, they're loosely the same thing, as you'll usually use the
Processing API to get an OpenGL window / context, but can dive deeper
into the OpenGL bindings by getting access to the low level interface.
But before considering that ...

> The most obvious way, as it seems to me, would be to write PImage-pixel data. But thats not possible as far as I can tell./!/? Also the texture should not be 8-bit-rgb, but float-grayscale. Texture buffer objects are preferred because of integer-indexing (but thats not really important)
>
> I think it would be great if I could manage to extend praxis-live in a way that shader.set(string "name", float or double array data) generates/updates a tbo.

You can definitely pass a PImage as a shader uniform. Assuming you've
correctly installed the additional custom components into the palette
(from the section on the start page), add a
video:gl:custom:glsl-io-transitions.pxg component to your graph, and
take a look at the code and shader in that. In particular, note the
uniforms -

uniform sampler2D from;
uniform sampler2D to;
uniform mat4 fromMatrix;
uniform mat4 toMatrix;

The matrix uniforms will be automatically set for you when using eg.
shader.set("from", fromImage);

Unfortunately, anything from a PImage will be 8-bit rgb as per
Processing defaults. Now we're maintaining our own Processing fork I
might look to extend that. At the moment, perhaps you can encode /
decode floats into the colour channels if need be?

Incidentally, Processing has a shader preprocessor that automatically
updates older GLSL code to newer - a shader with #version pragma will
bypass this stage.

Best wishes,

Neil

[Robert Oetting]

Hey Neil,

thank you for your thoughts and help!

Had not much time to work on this for a while, so my response is a bit delayed.

I am on the RGB-solution because its the most easy one and will work fine for now.

best regards,

Robert

--
You received this message because you are subscribed to the Google Groups "PraxisLIVE discussion group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to praxis-live...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/praxis-live/CAMYaZEvEhvJBA6PZKbVB-wAsik99ZbhcsMdN27kcv1kwX2vbFA%40mail.gmail.com.