Questions

[Nate Griswold]

Hello. I have a few questions:

1) Why do the docs say that racket_eval and racket_apply eval and apply in the “initial” or “original” racket thread? I have verified that the thread id returned from pthread_self when using racket_apply in c code called from a racket place is different from the thread id of the thread i called racket_boot on.

2) I noticed that if i racket_eval in c code called by a racket place, i can’t evaluate things that would otherwise (in racket code) be available in the place’s namespace. What is the correct way to get at things in the place’s racket namespace from c code? Is my problem unique to using a place, or would i have this problem in a non-place scenario as well?

3) (related to 2) I want to be able to put and get from a place channel from a long-running c function. If i just pass a place channel to the function, that is wrong because if i am correct in my thinking (and in what i have witnessed) there is a chance that when i go to use that channel in the future, it will have been moved by the garbage collector. Is there any way to prevent a specific value from being garbage collected, at least for the lifetime of my place? This is related to (2) because i actually don’t need this functionality if i can just racket_eval to get at the place channel from the namespace of the place’s module in the middle of a Sactivate_thread / Sdeactivate_thread session.

3) Is there any way to pass a c callback function into racket so that racket can call that function? I defined a ctype for the function, but i couldn’t think of a way to cast a pointer value passed into racket from c to an instance of my ctype (the `cast` function takes existing ctypes and not numeric values). Is there any way to manually make a value for my function ctype using an existing pointer value?

Regarding 2 and 3, i was able to solve the underlying problem behind those questions by creating a module to stash the channel in and passing its module path into c code, then importing it. I wonder if anyone has a better solution? Previously, i was using the `place` function for brevity.

Here is what i did:

(module myplace racket/base
 (provide place-main)
 (require syntax/location)

 (module stash racket/base
   (provide client-channel)
   (define client-channel (make-parameter #f)))

 (require 'stash)

 (define (place-main ch)
   (client-channel ch)
   (red_client_run_from_racket ch (quote-module-path stash))
   (error "Should not get here")))

(define p (dynamic-place (quote-module-path myplace) 'place-main))

and in c code:

int red_client_run_from_racket(ptr ch, ptr path) {
   ...
     racket_namespace_require(path);
   ...
}

Thank you

Nate

[Matthew Flatt]

At Mon, 23 Nov 2020 00:14:56 -0600, Nate Griswold wrote:
> Hello. I have a few questions:
>
> 1) Why do the docs say that racket_eval and racket_apply eval and apply in
> the “initial” or “original” racket thread? I have verified that the thread
> id returned from pthread_self when using racket_apply in c code called from
> a racket place is different from the thread id of the thread i called
> racket_boot on.

The `racket_apply` function was intended for use only in the OS thread
where Racket is booted, and only "outside" to get Racket started. It's
not intended for use in callbacks from Racket.

That's generally true for functions listed in sections 4-6 of "Inside",
and I see that the intent is not remotely clear there. I'll improve the
documentation.

To call back to Racket from C code, you should use the Racket FFI,
where a Racket function that's passed to C gets converted to a function
pointer for the C side. When you call that callback (as a regular C
function call), various bits of Scheme and Racket state get
configured/restored in a consistent way before jumping to the wrapped
Racket code as a callback.

For simple things, it turns out that `Scall0`, etc. would work to
invoke a callback within the Racket context that had called into C, but
there are many pitfalls, so you shouldn't do that.

You definitely should not use `racket_apply` or `racket_eval` from C
that was called from Racket. It will... well, do something. Take the
uncertainly of using `Scall0` and multiply by 100, since functions like
`racket_apply` specifically attempt to interact with the thread
scheduler.

> 2) I noticed that if i racket_eval in c code called by a racket place, i
> can’t evaluate things that would otherwise (in racket code) be available in
> the place’s namespace. What is the correct way to get at things in the
> place’s racket namespace from c code? Is my problem unique to using a
> place, or would i have this problem in a non-place scenario as well?

I think it's probably not just about places, but let me answer the
"correct way" question with the next bullet.

> 3) (related to 2) I want to be able to put and get from a place channel
> from a long-running c function. If i just pass a place channel to the
> function, that is wrong because if i am correct in my thinking (and in what
> i have witnessed) there is a chance that when i go to use that channel in
> the future, it will have been moved by the garbage collector. Is there any
> way to prevent a specific value from being garbage collected, at least for
> the lifetime of my place? This is related to (2) because i actually don’t
> need this functionality if i can just racket_eval to get at the place
> channel from the namespace of the place’s module in the middle of a
> Sactivate_thread / Sdeactivate_thread session.

The best approach here is to hand the C code a callback, which it will
see as plain old C functions. Maybe there's one callback to get from
the channel and another to put to the channel --- where the channel is
in the closure of the function, although the C side has no idea about
closures. If you go that route, then as long as you retain a reference
to the callback closure on the Racket side, the callback function
itself won't move.

Overall, reasoning about the interaction between Racket/Scheme and C
interaction from the C side is very difficult. The more you can arrange
for the C code to oblivious to Racket, the easier things get, because
the Racket-side tools for interacting with C are much better.

> 3) Is there any way to pass a c callback function into racket so that
> racket can call that function? I defined a ctype for the function, but i
> couldn’t think of a way to cast a pointer value passed into racket from c
> to an instance of my ctype (the `cast` function takes existing ctypes and
> not numeric values). Is there any way to manually make a value for my
> function ctype using an existing pointer value?

You can cast from a numeric value by casting from `_intptr` to a
pointer type. For example, this expression creates a function that
tries to jump to address 16 (and crashes, but in gdb you'd see it
crashing with the instruction pointer at address 16):

(cast 16 _intptr (_fun -> _void))


Matthew

[Nate Griswold]

Very helpful.

I have modified my code to use callbacks and everything is working as expected. It is interesting that everything was working fine with my racket_apply and racket_eval hacks, as well.

I have gotten my communication with racket down to one put and one get closure, wrapping a call to place-channel-put and place-channel-get. I think this is working well and the c client module is now mostly oblivious to racket, it just thinks it is using standard c function pointers so that is good. I do still have to Sactivate_thread and Sdeactivate_thread to construct my values and call the procs but that is ok, it is nice now.

The cast works properly for me, i thought i had tried that already but it works fine. I didn't realize `cast` works for source values that are already racket values but it makes sense that it could detect racket values and just not call c-to-racket, i guess.

Nate

[Nate Griswold]

Please ignore that last sentence.

Nate