Understanding how to display an Image and change pixels in real-time (fast)

[De Prins Maxime]

Hi,

I can do this:


using Base.Graphics

using Cairo

using Tk

using Base.Graphics

using Cairo

using Tk

win = Toplevel("Test", 800, 600)

c = Canvas(win)

pack(c, expand=true, fill="both")

ctx = getgc(c)

for x = 0:800

        for y = 0:600

               set_coords(ctx, x, y, 1, 1, 0, 1, 0, 1)

               set_source_rgb(ctx, x/800, y/600, 0)

               paint(ctx)

        end

end

reveal(c)

Tk.update()

But it is slow and not very nice.

Thanks

[Jameson Nash]

That's probably at least part of why Tk is horrible and needs to die a long overdue death. Whereas Gtk can simply share a memory buffer with the user with very little overhead. (full disclosure: I wrote and maintain the Gtk.jl wrapper, with the express purpose of making Tk go away)

julia> using Gtk.ShortNames

julia> win = @Window("Test", 800, 600);

julia> data = Gtk.RGB[Gtk.RGB(div(255*x,800),div(255*y,600),0) for x=0:800, y=0:600];

julia> pixbuf = @Pixbuf(data=data,has_alpha=false) # removing the has_alpha parameter required variable is https://github.com/JuliaLang/Gtk.jl/issues/94

julia> view = @Image(pixbuf);

julia> push!(win,view);

julia> showall(win);

julia> data[:] = Gtk.RGB(0,0,0);

julia> G_.from_pixbuf(view, pixbuf) # this function calls gtk_image_set_from_pixbuf, which is necessary to tell our GtkImage canvas that it needs to redraw now

julia> @time for i = 1:255

       data[:] = Gtk.RGB(0,0,i)

       G_.from_pixbuf(view, pixbuf)

       sleep(0.01)

       end

elapsed time: 3.799350398 seconds (113220 bytes allocated)

julia> 0.01*255

2.5500000000000003

[De Prins Maxime]

I don't care about using TK or GTK this is not a post about GTK or TK go to the julia-dev , What i  need is to understand how to change a Pixel somewhere on my screen.

I need to caclculate all pixels color one by one then display the screen (all the screen). I don't even need a windows (but maybe i have to)!

I am trying to understand cairo source code but i need help.

I don't know if the part i need is inside cairo or inside tk.

Just how to put a color somewhere on the screen.

Thanks you very mutch.

[De Prins Maxime]

Thanks you Jameson but i am here to learn and I woud like to see what's happening in details.

What code line is giving my color to my graphic cart and display the pixels

I wanna understand very closesly.

[Jameson Nash]

Cairo likely doesn't provide the right abstraction for coloring a pixel on the screen, since its focus is on drawing vector shapes. 

Similarly, Tk forces interaction with the library to be through strings, which is highly inefficient for passing in large chunks of data. 

Although if you only want to draw colors, and none of the other widgets that a toolkit provides, you might consider using one of the OpenGL bindings in Julia. 

[Tim Holy]

Jameson is correct in saying that for this project you need to abandon Tk and
use Gtk. As far as I can tell, Tk has no notion of updating a portion of a
window, so update has to redraw the entire window. That's why you get this:

On Monday, July 07, 2014 08:49:09 AM De Prins Maxime wrote:
> But it is slow and not very nice.

because it is very expensive to redraw the entire window just to change a
couple of pixels.

Gtk, on the other hand, seems to allow you to invalidate a portion of a window
and just redraw that. See this function:
https://github.com/JuliaLang/Gtk.jl/blob/master/src/events.jl#L100-L108
You're probably going to want to do something along the lines of what's in the
commented-out portion.

Also, while I'm not sure it's relevant for your use-case, one possible
resource is the gtk branch of ImageView, which exists to display images. The
top-level function is here:
https://github.com/timholy/ImageView.jl/blob/gtk/src/display.jl#L744-L747
You'll have to dig into the sub-functions yourself.

--Tim

[Simon Danisch]

Hi,
if you feel adventurous, you can try to use my OpenGL packages for this.
In a few days, this might even be a pleasant experience!

Here is how you would do this, right now:
https://gist.github.com/SimonDanisch/98aee37ddb76279cf774
still very messy, but this will ultimately be reduced to something like:

imgdata = texture([Vector4(0f0) for x=1:w, y=1:h])
gldisplay(imgdata)
#when you want to update:
imgdata[1:end,1:end] = [Vector4(float32(x/w), float32(y/h), 0f0, 1f0) for x=1:w, y=1:h]

These are the packages you need( I hope)
Pkg.add("Meshes")
Pkg.add("Images")
Pkg.add("ImmutableArrays")
Pkg.add("React")
Pkg.add("GLFW")
Pkg.clone("https://github.com/SimonDanisch/GLUtil.jl.git")
Pkg.clone("https://github.com/SimonDanisch/ModernGL.jl.git")
Pkg.clone("https://github.com/SimonDanisch/GLWindow.jl.git")

If you need ultimate performance and feel even more adventurous, you can try to implement your code in the shader.
Just change the shader to:

const fsh = """
in vec2 uv;
uniform sampler2D image;

uniform float customparam1; // if you need attributes from julia, put them here
uniform vec3 customparam2; // you can simply upload them like the texture in the render object
out vec4 fragment_color;
void main() {
	fragment_color = vec4(uv, 0, 1); //uv is already x/w, y/h
}
"""

You upload the custom params like this:

const fullscreenquad = RenderObject(
    [
        :position     => GLBuffer(GLfloat[-1,-1, -1,1, 1,1, 1,1, 1,-1, -1,-1], 2),
        :image         => image,
        :customparm1 => 1.0f0,

    ], GLProgram(vsh, fsh, "vertexshader", "fragmentshader"))

[Simon Danisch]

Damn, I accidentally hit post!
Well there is not much to add, besides that you can also use signals from React for the custom parameters:
customparam1 = Input(1f0)
customparam2 = Input(Vector3(0f0))

const fullscreenquad = RenderObject(
    [
        :position     => GLBuffer(GLfloat[-1,-1, -1,1, 1,1, 1,1, 1,-1, -1,-1], 2),
        :image         => image,

        :customparam1 => customparam1 ,
        :customparam2 => customparam2

    ], GLProgram(vsh, fsh, "vertexshader", "fragmentshader"))

and then you can update the parameters whenever you want like this:
push!(customparam1, 43942f0)
push!(customparam2, Vecto3(6f0,9f0,0f0))

I already excuse myself for any errors and inconsistencies, which you will encounter ;)
This is probably a little early to really use, but I was just writing some example code anyways...

[De Prins Maxime]

Maybe I have to describe better what I want to do:

My final purpose is to create a 3D world made of pixels (with a color and a 3D space coordinate),

then I want to explore this world with the mouse and the keybord.

------------------------------------------------------
So first I am trying to have the color control on each pixel in a window

but it has to be fast so in order to calculate all the visible pixels and display them on screen, and this several times per second

-----------------------------------------------

I am very glad I discoverd JULIA, the community is very cool and the language is awesome.

I am just a beginner, so it might take some time before I trully understand.

Thanks to all of you, I go back to work to test your ideas.

MAX

[Simon Danisch]

I see.
So the optimal solution would be to do everything inside a shader, or with OpenCL, as uploading a complete image to the framebuffer is extraordinary expensive.
Here's a demo of raytracing the Julia-Set with OpenCL and then displaying the resulting image in OpenGL, staying completely in video memory.
https://github.com/vchuravy/qjulia_gpu

Best,
Simon

[De Prins Maxime]

Thanks you very mutch I will look this closely in order to understand the basics.

I mean for exemple:

why a shader is fast and what it is...

how windows is using and talking with the graphic card...

what's happening in my computer....

and more important how do I write only the code line I need to do what i want  ( from nothing ) ...

;)

See you soon

[De Prins Maxime]

Ok , I know what I am going to do :).

So I have split my projet into 4 parts...

Part 1:

I need to find a way to store pixels in a 3D Euclidean space. (X,Y,Z)  with X,Y and Z INTEGER

I can do this with a 3xXxYxZ Array{Uint8,4}.

But I will need to know if a (X,Y,Z) position is empty or not (fast) and Ultimatly I woud like to have only the list of the pixels that exist really ( if the (X, Y, Z) position is empty I don't need to store this information into my data)

Part 2:

I need a fonction who will take two arguments and give me a picture.

Like this:

#pos is the position (x,y,z) of the camera and vec is the direction (x,y,z) looking by the camera

function camera_view(pos,vec)          
           #explore the data from part 1 and return a "3x600x800 Array{Uint8,3}" (for a 800x600 resolution screen)
end

Part 3:


For this part what i need is to create a loop that display and refresh on the screen the returned picture of the function from part 2

Something like this:

img = imread("mypicture.bmp")
display(img, pixelspacing = [1,1])

when (camera is moving)

               img.data = camera_view(pos,vec)
               display(img, pixelspacing = [1,1])
end

Part 4:

Use the keybord and the mouse to trigger the loop from part 3 and use the right (pos and vec) in the camera_view(pos,vec) function

-----------------------------The end------------------------------

I create this topic to answer the part3

so lets forget about all the other part.

My question transform to:

HOW CAN I DISPLAY x in a windows or using all the screen

like this:

LOOP
         x = rand(Uint8,3,800,600)
         Display (x)
END

and how can I refresh my windows/screen with a new x the fastest way (the point here is that I want to make a movie with random bitmaps)

idealy without using any Package/library  (writting all the code lines from nothing)

I am doing this to LEARN I know there are clever programmers outhere who made perfect tools for 3D.

I will post my work when I will get closer to the end haha.

Thanks again

[Tim Holy]

For 3D, Simon is the go-to person.

But for just displaying an image (which was your question at the bottom),
ImageView will do exactly what you want. See the README, focusing especially
on the "programmatic usage" section. If you're storing the array color-first,
you'll need to wrap it in an Image. See the Images package.

--Tim

On Tuesday, July 08, 2014 09:45:04 AM De Prins Maxime wrote:
> Ok , I know what I am going to do :).
>
> So I have split my projet into 4 parts...
>

> *Part 1:*

>
> I need to find a way to store pixels in a 3D Euclidean space. (X,Y,Z) with

> X,Y and Z *INTEGER*

>
> I can do this with a 3xXxYxZ Array{Uint8,4}.
>
> But I will need to know if a (X,Y,Z) position is empty or not (fast) and
> Ultimatly I woud like to have only the list of the pixels that exist really
> ( if the (X, Y, Z) position is empty I don't need to store this information
> into my data)
>
>
>

> *Part 2:*I need a fonction who will take two arguments and give me a

> picture.
>
> Like this:
>
> #pos is the position (x,y,z) of the camera and vec is the direction (x,y,z)
> looking by the camera
>
> function camera_view(pos,vec)

> #explore the data from *part 1* and return a "3x600x800

> Array{Uint8,3}" (for a 800x600 resolution screen)end
>
>
>
>

> *Part 3:*

> For this part what i need is to create a loop that display and refresh on
> the screen the returned picture of the function from
>

> *part 2*Something like this:

>
> img = imread("mypicture.bmp")
> display(img, pixelspacing = [1,1])
>
> when (camera is moving)
>
> img.data = camera_view(pos,vec)
> display(img, pixelspacing = [1,1])
> end
>
>
>
>
>

> *Part 4:*Use the keybord and the mouse to trigger the loop from *part 3*

> and use the right (pos and vec) in the camera_view(pos,vec) function
>
>
>

> *-----------------------------The end------------------------------*I

> create this topic to answer the part3
>
> so lets forget about all the other part.
>
> My question transform to:
>
>
>
>
>
>

> *HOW CAN I DISPLAY x in a windows or using all the screenlike
> this:LOOP x = **rand(Uint8,3,800,600)*
> *Display (x)*
> *END*

>
> and how can I refresh my windows/screen with a new x the fastest way
>

> * (the point here is that I want to make a movie with random bitmaps)*idealy

[De Prins Maxime]

Thanks tim, I made a wonderfull movie.
but you need to have a 800*600 image.bmp
to work properly

using Images

using ImageView

img = imread("image.bmp")

c = canvasgrid(1,1)

ops = [:pixelspacing => [1,1]]

display(c[1,1], img; ops...)

for i = 1:200

    img.data = rand(Uint8,3,800,600)

    display(c[1,1], img; ops...)

end

This is enought to start thinking about part1 and part2.

But i don't understand what happening in my computer :(

How to do this without using any package (What code lines of the images and imageview package I use)  , i will dig inside.

Max