Advice wanted about new opengl binding

[Hendrik Boom]

I need advice from someone with more experience with openGL and/or
Racket.

I am still working on updating the Racket openGL binding to the current
openGL spec.

I'm doing this as in the old binding, machine-translating the formal
part of the openGl standard, which now uses a competely new formalism.

I'm debugging it (for now) by comparing the old Racket include file with
the new one on the theory that functions that were there a few years ago and
are still there should have teh same Racket definitions now as then.

Running into a big snag.

There are a *lot* of glGet functions that grab data from the video
processor and bring it to the CPU.
They give openGL a pointer to a storage area where it should return
data.

For example, there's glGetCompressedTexImageARB. It receivees a
pointer to an image area and fills it in with a compressed image.
According the the Racket C interface, the way to write this
would be:

(define-gl glGetCompressedTexImageARB 2
( (target : _int32)
(level : _int32)
(img : _pointer/intptr)
-> _void -> img)
(->> exact-integer? exact-integer? gl-pointer?)
check-gl-error)

(here define-gl is a macro interal to the opengl binding that contains a
number of input arguments and a contract for the manual as well as the
actual c interface description, which of course is

( (target : _int32)
(level : _int32)
(img : _pointer/intptr)
-> _void -> img)

Here the '-> _void' specifies the value returned as function value from
C, and ->img indicates what the Racket openGL bindig s to return as to
the Racket calling program.

But the old binding instead used
( (target : _int32)
(level : _int32)
(img : _pointer/intptr)
-> _void)

In other words, it ignored the fact that img is used as an output
parameter.

Now what should I do with this? As I said there are very many such
functions. It's not an isolated case.

Are all these glGet functions simply something that never happened to
get used in Racket code so no one noticed the discrepancy?
Is there some idiom in Racket programming where all this just works
anyway?
Is there some efficency reason for not recognising output parameters?
Do they parhaps cause new storage allocation every time they are called?

I would prefer that other Racket code that uses the old binding not have
to be rewritten for the new. I'd like the new binding to be both
correct and compatible insofar as compatibility matters.

And I would very much prefer my binding to follow spec on both the
openGL front and the Racket front.

Or maybe I should call the new binding something else, like opengl3,
so that old code can use old bindings and new code the new.
That would be touble for those using old libraries who also want to use
ew features.

And I've encountered a few cases where I think the old binding is just
wrong.

I would appreciate advice on these matters from those with more Racket
or more openGL experience.

-- hendrik

[Matthew Flatt]

At Fri, 31 Jul 2020 17:05:27 -0400, Hendrik Boom wrote:
> For example, there's glGetCompressedTexImageARB. It receivees a
> pointer to an image area and fills it in with a compressed image.
> According the the Racket C interface, the way to write this
> would be:
>
> (define-gl glGetCompressedTexImageARB 2
> ( (target : _int32)
> (level : _int32)
> (img : _pointer/intptr)
> -> _void -> img)
> (->> exact-integer? exact-integer? gl-pointer?)
> check-gl-error)

If I understand correctly that `target` and `level` provide enough
information to specify how big `img` should be, then I agree that
something analogous to

(_fun
(target : _int32)
(level : _int32)
(img : _pointer/intptr = (malloc (COMPSIZE target level)))
-> _void
-> img)

is a fine way to wrap the function.

But when I work at the level of C APIs, I usually prefer this kind of
interface:

> But the old binding instead used
> ( (target : _int32)
> (level : _int32)
> (img : _pointer/intptr)
> -> _void)
>
> In other words, it ignored the fact that img is used as an output
> parameter.

Although, when I use this kind of binding, I have to know which
pointers must already be filled with information when I pass it to the
function versus which ones will be filled by the function, exposing the
pointer to to-be-filled memory gives me control over the way that the
memory. It also avoids having to specify anything about the
deallocation rules of the result of a binding (e.g., whether it needs
to be explicitly freed).

Still, I can also see the advantage of building allocation into the
wrapper to make clear that the argument corresponds to to-be-filled
memory.

Perhaps it depends on the overall shape of the API. If the API is going
to be wrapped further to make it more Rackety, then I'd go for less
wrapping at the immediate FFI bindings. If the API looks fairly Rackety
with a few tweaks to arguments and results, then I'd favor more
wrapping in the bindings. I think the old GL binding was in the former
camp: it first exposed GL functions in as raw as possible form, and
then wrapped those with a higher-level library.

> Now what should I do with this? As I said there are very many such
> functions. It's not an isolated case.
>
> Are all these glGet functions simply something that never happened to
> get used in Racket code so no one noticed the discrepancy?
> Is there some idiom in Racket programming where all this just works
> anyway?
> Is there some efficency reason for not recognising output parameters?
> Do they parhaps cause new storage allocation every time they are called?

I wouldn't say there's any "discrepancy" in `img` being an output
argument instead of an input argument. This kind of input--output
distinction doesn't exist at C ABI level, so no distinction is
inherently necessary in the FFI binding. Certainly, though, you can
create FFI wrappers that better reflect the intent of parameters than
the ABI-level types do.

You're right that exposing details like to-be-filled pointer arguments
usually offers the best available performance, but it's not always an
issue. This example looks like one where callers will practically
always allocate the destination memory just before calling the
function, so it will work out the same.


Matthew

[Hendrik Boom]

This makes it all feel like what the C documentation says about the function.
and you have to use it like C. And it helps a lot if you can know
whether the C code retains pointers to the storage area you passed it.
(Which I believe OpenGL does not, being organised entirely for passing
data to a different processor. But there could be problems if it the
data transfer were implemented using DMA asynchronously)

>
> Still, I can also see the advantage of building allocation into the
> wrapper to make clear that the argument corresponds to to-be-filled
> memory.

Yes. It's clear. But if usage is to allocate partucular storage areas
and use them repeatedly for getting data back from OpenGL, then this
iplementation is preferred.

>
> Perhaps it depends on the overall shape of the API. If the API is going
> to be wrapped further to make it more Rackety, then I'd go for less
> wrapping at the immediate FFI bindings. If the API looks fairly Rackety
> with a few tweaks to arguments and results, then I'd favor more
> wrapping in the bindings. I think the old GL binding was in the former
> camp: it first exposed GL functions in as raw as possible form, and
> then wrapped those with a higher-level library.

So for compatibility I should preserve the C-style calls.
Now it gets difficult. Some functions in the old bindings use what
seems to be the C-style conventions, and others the Racket-level
conventions with the two '->' arrows.

I have yet to figure out the conventions it uses.

I do know that the old specfiles from which the Racket binding was
machine-generated had an explicit keyword identifying output parameters.

The new ones in xml do not.

I'm going to have to investigate firther.

And track down the source code for both the C and Racket versions and
how they are generated. So far I've seen one.

And perhaps also the typed OpenGL function withing Pict3D.

This update isn't going to be done for quite a while unless I get some
inspiration.

It is going to take more thought. I don't want to create the situation
like with xml where there are two subtly incompatible implementations
being used, making it hard to use soem applications together.

(I'm using sxml to read the API definition. How that's the right
choice).

>
> > Now what should I do with this? As I said there are very many such
> > functions. It's not an isolated case.
> >
> > Are all these glGet functions simply something that never happened to
> > get used in Racket code so no one noticed the discrepancy?
> > Is there some idiom in Racket programming where all this just works
> > anyway?
> > Is there some efficency reason for not recognising output parameters?
> > Do they parhaps cause new storage allocation every time they are called?
>
> I wouldn't say there's any "discrepancy" in `img` being an output
> argument instead of an input argument. This kind of input--output
> distinction doesn't exist at C ABI level, so no distinction is
> inherently necessary in the FFI binding. Certainly, though, you can
> create FFI wrappers that better reflect the intent of parameters than
> the ABI-level types do.
>
> You're right that exposing details like to-be-filled pointer arguments
> usually offers the best available performance, but it's not always an
> issue. This example looks like one where callers will practically
> always allocate the destination memory just before calling the
> function, so it will work out the same.

I wonder which model of output-parameter passing is most often used
in out current crop of OpenGL applications.

One potential kludge is to generate the binding, hen compare it function
by function with the old one and use the old version when it exists and
the new one when it doesn't. I'd very much like to avoid that kind of
kludge.

By the way. there are also a fair number of enum's that have different
values in the old and new specs. Ugly. I hope no one's openGL library
demands we use one and not ther other. Or that no one uses them.

When I started this I had no idea how difficult it was going to be.
Translate form XML? Easy! Hah.

I mean, I've written and Algol 68 compiler. But this?

-- hendrik

>
>
> Matthew
>

[Hendrik Boom]

Time to rethink everything before I go further.

So far I've found several opengl bindings.
There's opengl, documented here:
https://docs.racket-lang.org/opengl/index.html
There are sgl and sgl/gl, documented here:
https://docs.racket-lang.org/sgl/index.html
and there's a typed opengl hidden with in pict3.

But I cannot find sgl and sgl/gl in the index of packages
at https://pkgs.racket-lang.org/

Shouldn't they be there?
Are they there in disguise?
Abd where should I look for current source code?
The index is pretty good at identifying source code for other packages.

-- hendrik

[Philip McGrath]

Is this what you're looking for? https://pkgs.racket-lang.org/package/sgl

-Philip

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/20200802215123.iiqik4wpfusarcw4%40topoi.pooq.com.

[Hendrik Boom]

On Mon, Aug 03, 2020 at 02:01:16PM -0400, Philip McGrath wrote:
> Is this what you're looking for? https://pkgs.racket-lang.org/package/sgl
>
> -Philip

Yes, looks like it. Is it messing from the index for some good reason?

I'm not sure how the packaging works.

I end up at https://github.com/racket/sgl/tree/master
where I find multiple files, including main.rkt, sgl.rlt, and gl.rkt.
Am I correct that main.rkt is what I get with (require sgl)
and that gl.rkt is what I get with (require sgl/gl)?

-- hendrik

> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAH3z3gYmn4_5sTUNWuZDcpjL5shJmU2quTtgx72%3DoycOXccp5A%40mail.gmail.com.

[Sam Tobin-Hochstadt]

On Mon, Aug 3, 2020, 7:27 PM Hendrik Boom <hen...@topoi.pooq.com> wrote:

On Mon, Aug 03, 2020 at 02:01:16PM -0400, Philip McGrath wrote:
> Is this what you're looking for? https://pkgs.racket-lang.org/package/sgl
>
> -Philip

Yes, looks like it.  Is it messing from the index for some good reason?

By default, the front page of pkgs.racket-lang.org doesn't show packages that are in the main-distribution, such as sgl. 

I'm not sure how the packaging works.

I end up at https://github.com/racket/sgl/tree/master
where I find multiple files, including main.rkt, sgl.rlt, and gl.rkt.
Am I correct that main.rkt is what I get with (require sgl)
and that gl.rkt is what I get with (require sgl/gl)?

That's correct. 

Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/20200803232728.e4yv7khscvr7c2wl%40topoi.pooq.com.