GC of closures

[Mike Jones‏]

Ref 3.13 of Introduction, ifold2...

Do closures of in general:

- Gen code with a malloc call?
- Gen code with free after exiting scope?

Meaning, if I am not linking a library with malloc and free I can't accidentally use it, and if I do have a library, I can't accidentally produce a leak?

For example, the function example sqrmodsum in the text: does it produce an error when malloc is not available, and if available, does this example leak? Or does it make one heap object that is reused on each call?

I guess I want to know under what circumstances a nieve maintainer can get into trouble. My hope is the behavior is either fail to compile, or no leak. If I am correct, then in which ways can I create a closure that leaks so I can r cognizant them?

[Mike Jones‏]

A related question is why wouldn't the example sqrmodsum put the closure on the stack? It is not needed outside the scope of this function as it is not in any way returned, and there is no lazy evaluation.

[gmhwxi‏]

In general, malloc needs to be called to construct a closure.
However, 'free' is *not* called automatically. If you use a linear
closure (cloptr) but do not call 'free', you will get a type-error somewhere.
If you use a persistent closure (cloref), you can not free the closure explicitly
(in a type-safe manner). Instead, GC is needed to reclaim the closure.

In other words, if you create a cloref but do not run GC, you have a potential
memory leak.

[gmhwxi‏]

To recognize whether the C code generated from ATS source
does something bad (e.g., leaks memory), one needs to analyze the
C code. Tools need to be developed for doing such analysis.

[gmhwxi‏]

Yes, you can allocate the closure on the stack:

https://github.com/githwxi/ATS-Postiats-test/blob/master/contrib/hwxi/TEST30/test31.dats

Note that this is static allocation; it is *not* done by alloca.

[Yannick Duchêne‏]

Le lundi 28 décembre 2015 23:13:53 UTC+1, Mike Jones a écrit :

Meaning, if I am not linking a library with malloc and free I can't accidentally use it, and if I do have a library, I can't accidentally produce a leak?

 

Le mardi 29 décembre 2015 01:30:35 UTC+1, gmhwxi a écrit :

To recognize whether the C code generated from ATS source
does something bad (e.g., leaks memory), one needs to analyze the
C code. Tools need to be developed for doing such analysis.

I can't find back the thread, I just remember exceptions may produce memory leaks too. 

[Mike jones‏]

I assume that if there is no alloc/malloc library to link, there cannot be links, because I kind of remember that for ATS the GC operates via this API. I would like to confirm that.

The danger for me is my application layers on ThreadX, wrappers by Cypress, but I think it exposes the alloc/malloc. Would a simple search of the generated C be enough to determine no leaks? Or are there other calls I need to check?

Sent from my iPad

--
You received this message because you are subscribed to a topic in the Google Groups "ats-lang-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ats-lang-users/zht_Kex7y-0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.
Visit this group at https://groups.google.com/group/ats-lang-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/eba42b74-d4a6-4d89-987d-a244dbccd36a%40googlegroups.com.

[Hongwei Xi‏]

To use malloc, you need to compile with a flag looking like -DATS_MEMALLOC_????

For instance,

patscc -DATS_MEMALLOC_LIBC -o foo foo.dats // using malloc/free in libc

patscc -DATS_MEMALLOC_GCBDW -o foo foo.dats -lgc // using Bohem GC

If you use malloc but do not use -DATS_MEMALLOC_????, then you will see

the following error message issued by the linker:

undefined reference to `atsruntime_malloc_undef'

If you can compile your ATS code to executable without using -DATS_MEMALLOC_????,

then your ATS code cannot generate memory leaks (because it does not even call malloc).

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-user...@googlegroups.com.

To post to this group, send email to ats-lan...@googlegroups.com.
Visit this group at https://groups.google.com/group/ats-lang-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/EEAA785F-0807-48A8-8126-18A012B9CBDA%40gmail.com.

[gmhwxi‏]

Try to compile the one calls 'malloc' and the one that does not, and you will
see the difference clearly:

https://github.com/githwxi/ATS-Postiats-test/blob/master/contrib/hwxi/TEST30/test31.dats

On Tuesday, December 29, 2015 at 10:10:44 AM UTC-5, gmhwxi wrote:

To use malloc, you need to compile with a flag looking like -DATS_MEMALLOC_????

For instance,

patscc -DATS_MEMALLOC_LIBC -o foo foo.dats // using malloc/free in libc

patscc -DATS_MEMALLOC_GCBDW -o foo foo.dats -lgc // using Bohem GC

If you use malloc but do not use -DATS_MEMALLOC_????, then you will see

the following error message issued by the linker:

undefined reference to `atsruntime_malloc_undef'

If you can compile your ATS code to executable without using -DATS_MEMALLOC_????,

then your ATS code cannot generate memory leaks (because it does not even call malloc).

On Tue, Dec 29, 2015 at 9:56 AM, Mike jones <proc...> wrote:

I assume that if there is no alloc/malloc library to link, there cannot be links, because I kind of remember that for ATS the GC operates via this API. I would like to confirm that.

The danger for me is my application layers on ThreadX, wrappers by Cypress, but I think it exposes the alloc/malloc. Would a simple search of the generated C be enough to determine no leaks? Or are there other calls I need to check?

Sent from my iPad

On Dec 28, 2015, at 5:30 PM, gmhwxi <gm...> wrote:

To recognize whether the C code generated from ATS source
does something bad (e.g., leaks memory), one needs to analyze the
C code. Tools need to be developed for doing such analysis.

On Monday, December 28, 2015 at 7:28:14 PM UTC-5, gmhwxi wrote:

In general, malloc needs to be called to construct a closure.
However, 'free' is *not* called automatically. If you use a linear
closure (cloptr) but do not call 'free', you will get a type-error somewhere.
If you use a persistent closure (cloref), you can not free the closure explicitly
(in a type-safe manner). Instead, GC is needed to reclaim the closure.

In other words, if you create a cloref but do not run GC, you have a potential
memory leak.

On Monday, December 28, 2015 at 5:13:53 PM UTC-5, Mike Jones wrote:

Ref 3.13 of Introduction, ifold2...

Do closures of in general:

- Gen code with a malloc call?
- Gen code with free after exiting scope?

Meaning, if I am not linking a library with malloc and free I can't accidentally use it, and if I do have a library, I can't accidentally produce a leak?

For example, the function example sqrmodsum in the text: does it produce an error when malloc is not available, and if available, does this example leak? Or does it make one heap object that is reused on each call?

I guess I want to know under what circumstances a nieve maintainer can get into trouble. My hope is the behavior is either fail to compile, or no leak. If I am correct, then in which ways can I create a closure that leaks so I can r cognizant them?

--
You received this message because you are subscribed to a topic in the Google Groups "ats-lang-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ats-lang-users/zht_Kex7y-0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to ats-lang-users+unsubscribe@googlegroups.com.
To post to this group, send email to ats-lang-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ats-lang-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/eba42b74-d4a6-4d89-987d-a244dbccd36a%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-users+unsubscribe@googlegroups.com.
To post to this group, send email to ats-lang-users@googlegroups.com.

[Mike Jones‏]

Since Bohem GC replaces alloc, does the ATS compiler really generate different code for GC or does it just add some GC initiallization code?

[Hongwei Xi‏]

ATS compiler generates the same code. It is just that certain functions (malloc, free, calloc, and realloc) are

hooked to different implementations (provided by different libraries).

On Tue, Dec 29, 2015 at 12:10 PM, Mike Jones <proc...@gmail.com> wrote:

Since Bohem GC replaces alloc, does the ATS compiler really generate different code for GC or does it just add some GC initiallization code?

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.
Visit this group at https://groups.google.com/group/ats-lang-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/c14d350a-f275-4fa0-9eae-7586e59d850f%40googlegroups.com.

[Mike Jones‏]

I need some help with this code:

=<clo1> I assume overrides a default of <cloref1>

lam@ I'm not sure what @ means. Does it mean pass tuple, anything, is something non-transparent being made transparent?

Cast does not use {}, why not? Is the compiler figuring it out on its own? If so, should it be {ptr} if explicit?

Cast because by not being ref, it has to be converted to a ptr? Seem messy not to just create a closure as pointer in the first place, so that type checking does its job.

fun{}
sqrmodsum2
(n: int, d: int): int = let
//
var
fopr =
lam@(res: int, x: int): int =<clo1>
if x mod d = 0 then res + x * x else res
//
in
ifold2(10, $UNSAFE.cast(addr@fopr), 0)
end // end of [sqrmodsum2]

[Mike Jones‏]

This shows how to put it on heap/GC, and stack, how do you put it in static memory? I assume you create a standard function or instantiated template. Then do you have to cast it?

[Hongwei Xi‏]

I should have said 'stack-allocated' (instead of 'statically allocated'). However, the stack

allocation in the original call-frame of the function; it is not done via alloca.

On Tue, Dec 29, 2015 at 12:34 PM, Mike Jones <proc...@gmail.com> wrote:

This shows how to put it on heap/GC, and stack, how do you put it in static memory? I assume you create a standard function or instantiated template. Then do you have to cast it?

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.

Visit this group at https://groups.google.com/group/ats-lang-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/9206989e-1eaa-46e4-927d-541b9b4df898%40googlegroups.com.

[Barry Schwartz‏]

On Tue, Dec 29, 2015 at 12:10 PM, Mike Jones <proc...@gmail.com> wrote:
> Since Bohem GC replaces alloc, does the ATS compiler really generate
> different code for GC or does it just add some GC initiallization code?

Hongwei Xi <gmh...@gmail.com> skribis:

> ATS compiler generates the same code. It is just that certain functions
> (malloc, free, calloc, and realloc) are
> hooked to different implementations (provided by different libraries).

Which works because on most architectures Boehm GC needs no
initialization.

The same would not be true if one used the Boehm GC wrappers in
libguile, for instance. I’m curious if there is a way to be sure
initialization is called before dynloads.

[Barry Schwartz‏]

Barry Schwartz <chemoe...@chemoelectric.org> skribis:

Urgh, that was probably a silly question. I would imagine just be sure
to put it in the first thing dynloaded.

[Mike Jones‏]

There is another closure example in the introduction book I want to understand. Section 6.5 has code below.

The fun definitions have <cloref1>, and do not follow the typical syntax of a value bound to a lam. So does this mean a closure is returned? Does it mean the fun is a closure? Is any thing on the heap? And what is the purpose of it in this example?

fun{

a:t@ype

} insertion_sort

(

A: arrszref (a)

, cmp: (a, a) -> int

) : void = let

val n = g0uint2int_size_int(A.size())

fun ins (x: a, i: int):<cloref1> void =

if i >= 0 then

(

if cmp (x, A[i]) < 0

then (A[i+1] := A[i]; ins (x, i-1)) else A[i+1] := x

// end of [if]

) else A[0] := x // end of [if]

// end of [ins]

fun loop (i: int):<cloref1> void =

if i < n then (ins (A[i], i-1); loop (i+1)) else ()

// end of [loop]

in

loop (1)

[gmhwxi‏]

'lam@' means to form a closure in a given memory space.

In order words, any closure created with the keyword 'lam@' is flat.

I was cutting a bit of corner in my original code. No casting is needed if
ifold2 is replaced with ifold2_:

fun
ifold2_( n: int, f: &(int, int) -<clo1> int, ini: int) : int =
  if n > 0 then f(ifold2_ (n-1, f, ini), n) else ini// end of [ifold2]

[gmhwxi‏]

The initialization function for Bohem-GC is called by atsruntime_minit_gcbdw:

ATSinline()
atsvoid_t0ype
atsruntime_minit_gcbdw
  (/*void*/) { GC_init() ; return ; }
// end of [atsruntime_minit_gcbdw]

If you use -DATS_MEMALLOC_GCBDW, atsruntime_minit_gcbdw is automatically
called.

[gmhwxi‏]

The annotation '<cloref1>' means that IF the function is passed as an argument
(to another function), then a closure is formed. If the function is never passed as
an argument, there is no closure formation.

This code is of ATS1-style. I think '<cloref1>' can be erased here without causing
any problems.

[Barry Schwartz‏]

gmhwxi <gmh...@gmail.com> skribis:

> The initialization function for Bohem-GC is called by
> atsruntime_minit_gcbdw:
>
> ATSinline()
> atsvoid_t0ype
> atsruntime_minit_gcbdw
> (/*void*/) { GC_init() ; return ; }
> // end of [atsruntime_minit_gcbdw]
>
> If you use -DATS_MEMALLOC_GCBDW, atsruntime_minit_gcbdw is automatically
> called.

Now I realize it was a REALLY silly question, because I KNEW there
were minit functions. :)

[Mike Jones‏]

What does the & mean?

[Hongwei Xi‏]

call-by-reference.

On Tue, Dec 29, 2015 at 3:23 PM, Mike Jones <proc...@gmail.com> wrote:

What does the & mean?

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.

Visit this group at https://groups.google.com/group/ats-lang-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/8d8cb85b-f530-4d07-9dec-a16e35c3fc19%40googlegroups.com.

[gmhwxi‏]

It is kind of like:

typedef clo *cloref;

On Tuesday, December 29, 2015 at 3:26:27 PM UTC-5, gmhwxi wrote:

call-by-reference.

On Tue, Dec 29, 2015 at 3:23 PM, Mike Jones <pro...> wrote:

What does the & mean?

--
You received this message because you are subscribed to the Google Groups "ats-lang-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-users+unsubscribe@googlegroups.com.
To post to this group, send email to ats-lang-users@googlegroups.com.

[Mike Jones‏]

So (...) -<cloref1> means closure on heap which is ptr, and &(...) -<clo1> means ptr to closure that is not already ptr, on stack or static. Wouldn't the representation be the same, such that a function taking a closure as argument could take either, assuming something like ptr to tuple with an anonymous function and captured data?

And can you pass a closure by value? That assumes a tuple copy. Not efficient, but curious.

[gmhwxi‏]

Yes, the same representation is used. That is why I used '$UNSAFE.cast'
in my original code.

Passing a closure by value is *impossible* in general because the type 'clo'
is not of a fixed size (as we cannot tell anything about the environment in a closure).

[Yannick Duchêne‏]

Le mardi 29 décembre 2015 19:26:54 UTC+1, gmhwxi a écrit :

'lam@' means to form a closure in a given memory space.

Excuse my question: does that mean one can use statically allocated closures, I mean, closure without any dynamic memory allocation? 

[Yannick Duchêne‏]

Please Professor, can you add the “documentation” tag to this thread?

[Hongwei Xi‏]

Yes, absolutely.

--

You received this message because you are subscribed to the Google Groups "ats-lang-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.

Visit this group at https://groups.google.com/group/ats-lang-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ats-lang-users/b50a3ae2-d7f1-4981-b40b-8cfded0be221%40googlegroups.com.

[Mike Jones‏]

Can you add the cast version in the code or write it here? I think it is important because one might need to pass stack allocated lams to prexisting higher ordered functions. As much as I don't like casting, for embedded code, this seems the only way, unless there is something not said here.

[Mike Jones‏]

BTW, how does the compiler know what type to cast to?

[gmhwxi‏]

If the type of the target of cast is not given, then the typerchecker
does not check anything. So this kind of cast is the most unsafe kind.

[gmhwxi‏]

Here is the code making use of unsafe casting:

//
// HX-2015-12-31:
// this one need a cast:
//
fun{}
sqrmodsum
  (n: int, d: int): int = let

//
var
fopr =
lam@(res: int, x: int): int =<clo1>
  if x mod d = 0 then res + x * x else res
//
in
  ifold2(10, $UNSAFE.cast(addr@fopr), 0)

end // end of [sqrmodsum]
//