Re: [go-nuts] Unset / Free memory of an entire Map or any variable type ?

[Jan Mercl]

On Mon, Apr 15, 2013 at 2:00 PM, <rahe...@gmail.com> wrote:
> I have been looking for this for sometime now. But is it seriously not
> possible to delete / unset / free a variable or a part of memory ?
> As per the spec there is only one function delete to delete the key of a map
> ?
> But what to do I would like to free the value of strucure element
> Lets say I have a structure :
> type MyStruct struct {
> d []int
> v []int
> }
>
> Now if I have to reset MyStruct.d would a simple make clear the memory ?
> If not, then how should I clear the memory so that I can ensure no memory
> leak.

Go is a garbage collected language. In normal Go programs there's thus
no way how to explicitly free memory. To make memory eligible for
collection, it must get unreachable (eg. make all existing pointers to
something equal to nil), which is as far as you can get with explicit
memory "control".

-j

[chris dollin]

On 15 April 2013 13:00, <rahe...@gmail.com> wrote:

I have been looking for this for sometime now. But is it seriously not possible to delete / unset / free a variable or a part of memory ?

That's what garbage-collection is for.
 

As per the spec there is only one function delete to delete the key of a map ?
But what to do I would like to free the value of strucure element
Lets say I have a structure :

type MyStruct struct {

d []int
v []int

}

If you're done with it (and there are no more references to it), you don't

/need/ to free it (although there are cases where you want to keep it

around and re-use it. But I think all you'd need here is

    mystruct = MyStruct{}

Chris

--
Chris "allusive" Dollin

[Johann Höchtl]

What about

mystruct = nil ?

[Jan Mercl]

On Mon, Apr 15, 2013 at 9:42 PM, Johann Höchtl <johann....@gmail.com> wrote:
>> If you're done with it (and there are no more references to it), you don't
>> /need/ to free it (although there are cases where you want to keep it
>> around and re-use it. But I think all you'd need here is
>>
>> mystruct = MyStruct{}
>
> What about
>
> mystruct = nil ?

'nil' is not assignable to 'mystruct' of type 'MyStruct' (it's not a
pointer type).

-j

[Hamish Ogilvy]

If something is big and going to be created and deleted a lot, i would put it in it's own struct and use a pointer to reference it, that way you know when the pointer is removed, the GC will collect it. So from a design point of view, think about what will be using a lot of memory and at some point needs to be freed, isolate it into its own struct and use pointers to the struct. If v and d in your example were struct pointers themselves, you would have much better control.

Also take note of the slice tricks at the following URL. Things like removing one element from a slice can easily end up copying the whole slice, so be aware of techniques to minimise memory duplication also.

http://code.google.com/p/go-wiki/wiki/SliceTricks

On Monday, 15 April 2013 22:00:17 UTC+10, rahe...@gmail.com wrote:

I have been looking for this for sometime now. But is it seriously not possible to delete / unset / free a variable or a part of memory ?

As per the spec there is only one function delete to delete the key of a map ?
But what to do I would like to free the value of strucure element
Lets say I have a structure :

type MyStruct struct {

d []int
v []int

}

[Raheel Gupta]

>> To make memory eligible for collection, it must get unreachable

Even when this is done, you still have to wait for the GC to do its stuff ? Or not. If you have to wait for the GC to free it up, memory intensive applications will be a difficult thing to write in GO and hacking GO based servers or codes would be easy as one could easily break the software with multiple connections ?

@Hamish, but not being able to wilfully free memory makes GO one weak coding software.

Why does any software need to be so reliant on the GC. The GC is a good thing but I believe the coder should also be given the right to free memory. Also when no pointer points to a block of memory, will it be freed instantaneously ?

[Dan Kortschak]

I regularly write code that uses upwards of 60-100GB in Go, with massive
heap growth and reduction (largely not due to ineptitude ;), but rather
the shape of the data we deal with). I don't find there is any problem
due to a lack of an explicit Free().

[Raheel Gupta]

If you're done with it (and there are no more references to it), you don't

/need/ to free it (although there are cases where you want to keep it

around and re-use it. But I think all you'd need here is

    mystruct = MyStruct{}

What if I am allocating memory in 100MB per loop and I have to free it at the end of the loop. If you say, just leave it to the GC, by the time I have done a 100 loops the program would have crashed right ?

[Dave Cheney]

No, you would have overwritten mystruct 100 times.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Hamish Ogilvy]

I used to think the same thing, but as with Dan, i'm less worried about it now. We use up to ~30GB in production, the GC causes an occasional pause (using Go 1.03, 1.1 is meant to be better), but otherwise it's fine. It makes you think a bit more when coding to re-use and minimise memory allocation. Of course there is an overhead as the GC is periodic not instantaneous, but if you're having memory run away issues, i would be very surprised if it couldn't be fixed in your code.

[Rémy Oudompheng]

2013/4/16 Raheel Gupta <rahe...@gmail.com>:

No, the GC runs before you do 100 loops.

Rémy.

[Raheel Gupta]

On Tue, Apr 16, 2013 at 10:56 AM, Dave Cheney <da...@cheney.net> wrote:

No, you would have overwritten mystruct 100 times.

But in the last loop it would be left with 100 MB of Garbage memory till it was collected by the GC right ?

[Dan Kortschak]

On Tue, 2013-04-16 at 10:54 +0530, Raheel Gupta wrote:
> What if I am allocating memory in 100MB per loop and I have to free it at
> the end of the loop. If you say, just leave it to the GC, by the time I
> have done a 100 loops the program would have crashed right ?

Nope.

[Dave Cheney]

I think you are getting off on the wrong foot. GC has been around since LISP in the 50's, all these problems have been discussed, understood and solved decades ago. Java is garbage collected, ruby is garbage collected, JavaScript is garbage collected, etc. 

This is both not something to worry about, and also not something you can do anything about, Garbage collecting is integral to Go. 

[Raheel Gupta]

I think you are getting off on the wrong foot. GC has been around since LISP in the 50's, all these problems have been discussed, understood and solved decades ago. Java is garbage collected, ruby is garbage collected, JavaScript is garbage collected, etc. 

This is both not something to worry about, and also not something you can do anything about, Garbage collecting is integral to Go. 

 

 

Maybe, but I still feel having an option to free memory anytime alongwith the GC would be a better solution. I love the GC, but something to free the memory would always be handy.

 

[Dan Kortschak]

On Tue, 2013-04-16 at 11:36 +0530, Raheel Gupta wrote:
> I love the GC, but something to free the memory would always be handy.

Why? If it's freed by the GC or you, does it make any real difference?
You can't get Go without GC, so you will always have the the GC
overhead. What do you get from the capacity to free objects manually?

[Dave Cheney]

You can't free the memory; you don't own it, the runtime does. You cant free memory without potentially creating a dangling pointer. 

If you absolutely know that nobody else has a reference to that memory, then good news, so does the compiler, and it has already allocated you memory on the stack via escape analysis and it will be freed when you return from your function. 

[Volker Dobler]

There are plenty of languages which offer manual memory management:

Alef, Brainfuck, C, ... to name some.

Seriously: You did not demonstrate that your code really does run oom

or has any other issue because you cannot manually free allocated memory.

Most probably this lack of evidence shows that manually freeing simply

is not necessary and really not worth a sleepless night.

V.

[Peter]

In a recent article published by Google, the authors briefly mentioned controlling the latency caused by garbage collected languages. http://cacm.acm.org/magazines/2013/2/160173-the-tail-at-scale/fulltext

Since Go was (originally) designed with these kind of programs in mind, I wonder if the current control we have over the GC is enough. As I understand, we can trigger it to run (runtime.GC()), but not prevent it from running.

[Carlos Castillo]

http://tip.golang.org//pkg/runtime/debug/#SetGCPercent If you set it from the default of 100, to a negative value, the collector doesn't run on its own. This is generally not a good idea though, as your performance gains before running out of real memory will be minimal. If you make a program that doesn't fill up all available memory because it re-uses values, and minimizes allocations, that code will be just as fast with the collector on as off, since the collector won't be running often (or at all).

[Kevin Gillette]

On Tuesday, April 16, 2013 12:06:36 AM UTC-6, Raheel Gupta wrote:

Maybe, but I still feel having an option to free memory anytime alongwith the GC would be a better solution. I love the GC, but something to free the memory would always be handy.

For now, seasoned gophers are pretty good at avoiding garbage where it matters, and not caring where it doesn't. See http://blog.golang.org/2011/06/profiling-go-programs.html.

Just like you should profile to make sure you need it before you spend effort optimizing, you should profile before you come to the conclusion that you need to explicitly free things. One of the strategies that has kept Go from mutating into C++ is to avoid extending the language or standard libs with anything that could eventually be done by the runtime or the compiler, even though it may need suboptimal code is being generated now.

Looking to the future, there are a great many tangible compiler optimization opportunities, even within the current mark/sweep collector approach (other collectors have been discussed at length elsewhere):

• Fixed-size allocations, such as `new(struct{...})`, which don't escape are currently allocated on the stack.
• Variable-size allocations (slices, maps), which don't escape could be freed on stack pop.
• Running functions that don't access non-local memory would not need to stop for garbage collection. They would only need to pause at a stack boundary when:

1. Calling into a function that shares memory
2. Unrolling back into a function that shares memory
3. (the bottom/frozen portion of a goroutine stack could thus be scanned even while the top of the same stack is 'hot')
4. Channels holding fixed size "value" data could also count as non-escaping for the above purpose, and the channels themselves could get allocated in a delayed collection arena.

Besides, giving people the ability to free memory would lead to abuse; for one thing, we'd get mailing list questions asking why the compiler wouldn't allow `var x int; free(x)`.