C++ and pointer registration with 3M?

[Thomas Dickerson]

The documentation says:

"The 3m collector needs to know the address of every local or temporary pointer within a function call at any point when a collection can be triggered."

A couple questions:

- Is this "every pointer which might be passed to a Racket function, or contain the result of a Racket function"? Or is it truly every pointer?

- Do I need to register the data pointer passed to `scheme_main_stack_setup`, even though this is above the GC's stack base?

- In an instance method, does `this` need to be registered? What about reference types?

[Thomas Dickerson]

Sorry for the double-post, one more question.

The output of `raco ctool --c-mods` has the form:

#ifdef MZ_XFORM
XFORM_START_SKIP;
#endif
static void declare_modules(Scheme_Env *env) {
  static unsigned char data[] = { /* bytes go here */ };
  scheme_register_embedded_load(NUM_BYTES, (const char *)data);
  scheme_embedded_load(NUM_BYTES, (const char *)data, 1);
}
#ifdef MZ_XFORM
XFORM_END_SKIP;
#endif

As I understand it, this means there's no reason to pass it through `raco ctool --xform`, which presumably means we don't need need to register `env` or `data`, but we are seeing `scheme_embedded_load` in the stack trace of some Racket routines that triggered a watchpoint on memory that it shouldn't be touching, so I just wanted to double check that's indicative of a problem upstream.

[Matthew Flatt]

At Fri, 1 Nov 2019 10:34:50 -0700 (PDT), Thomas Dickerson wrote:
> The documentation says:
> "The 3m collector needs to know the address of every local or temporary
> pointer within a function call at any point when a collection can be
> triggered."
>
> A couple questions:
> - Is this "every pointer which might be passed to a Racket function, or

> contain the result of a Racket function"? Or is it truly *every pointer*?

Only pointers to GCable memory --- which tends to be the things passed
to a Racket function, but more generally corresponds to things that
come from Racket .

> - Do I need to register the data pointer passed to
> `scheme_main_stack_setup`, even though this is above the GC's stack base?

No. As you say, that's before the GC is ready.

> - In an instance method, does `this` need to be registered? What about
> reference types?

A `this` pointer does not point to GCable memory, unless you do well
out of your way to allocate an object with Racket's allocator.

The value of a variable with a reference type would normally correspond
to an address on the C stack, so it would not need to be registered.

At Fri, 1 Nov 2019 11:41:18 -0700 (PDT), Thomas Dickerson wrote:
> The output of `raco ctool --c-mods` has the form:
>
> #ifdef MZ_XFORM
> > XFORM_START_SKIP;
> > #endif
> > static void declare_modules(Scheme_Env *env) {
> > static unsigned char data[] = { /* bytes go here */ };
> > scheme_register_embedded_load(NUM_BYTES, (const char *)data);
> > scheme_embedded_load(NUM_BYTES, (const char *)data, 1);
> > }
> > #ifdef MZ_XFORM
> > XFORM_END_SKIP;
> > #endif
> >
>
> As I understand it, this means there's no reason to pass it through `raco
> ctool --xform`, which presumably means we don't need need to register `env`
> or `data`, but we are seeing `scheme_embedded_load` in the stack trace of
> some Racket routines that triggered a watchpoint on memory that it
> shouldn't be touching, so I just wanted to double check that's indicative
> of a problem upstream.

The `data` pointer is ok because it refers to statically allocated
memory, which is not managed by the GC. The `env` pointer isn't
registered because it isn't used, so it's ok if `env` gets out of date.

[Thomas Dickerson]

Great, this is very helpful.

On Fri, Nov 1, 2019 at 4:53 PM Matthew Flatt <mfl...@cs.utah.edu> wrote:

At Fri, 1 Nov 2019 10:34:50 -0700 (PDT), Thomas Dickerson wrote:

Only pointers to GCable memory --- which tends to be the things passed

to a Racket function, but more generally corresponds to things that
come from Racket .

Is it an error to register things which are not GCable, or just unnecessary?

Also - are there ever likely to be pointers which could hold memory which may or may not be GCable depending on the code path?

(e.g. a `const char*` that is sometimes converted from a Racket string, and sometimes provided by a C string)

> - Do I need to register the data pointer passed to
> `scheme_main_stack_setup`, even though this is above the GC's stack base?

No. As you say, that's before the GC is ready.

> - In an instance method, does `this` need to be registered? What about
> reference types?

A `this` pointer does not point to GCable memory, unless you do well
out of your way to allocate an object with Racket's allocator.

The value of a variable with a reference type would normally correspond
to an address on the C stack, so it would not need to be registered.

Great, this matches my mental model of what should be happening, but I've been seeing very strange behavior, so I was getting paranoid.

 

[Matthew Flatt]

At Fri, 1 Nov 2019 17:15:57 -0400, Thomas Dickerson wrote:
> Is it an error to register things which are not GCable, or just unnecessary?

Just unnecessary.

> Also - are there ever likely to be pointers which could hold memory which
> may or may not be GCable depending on the code path?
> (e.g. a `const char*` that is sometimes converted from a Racket string, and
> sometimes provided by a C string)

It's not common, but that can happen. The GC can recognize and ignore
pointers that are not to its own space.

[Thomas Dickerson]

On Fri, Nov 1, 2019 at 6:04 PM Matthew Flatt <mfl...@cs.utah.edu> wrote:

Just unnecessary.

Great :)

It's not common, but that can happen. The GC can recognize and ignore
pointers that are not to its own space.

This also fits my mental model, but I wanted to make sure I understood the corner cases.

Thanks!

--

Thomas Dickerson

Founder // Chief Science Officer

Geopipe, Inc.

802-458-0637