I have a long running Go program that allocates a large amount of
memory over time. On FreeBSD 64 bit I am seeing a problem where once I
am done with the memory the RES size as reported by top never
decreases.
For example, after my program has finished processing requests I stop
all requests and let CPU drop to 0% and then leave it so that GC and
scavenging has time to work. Then I look at the RES size in top and
dump out a memory profile for pprof. In one example, I have RES of
15666M (i.e. 15GB) and pprof showing me:
(pprof) top10
Total: 32.7 MB
This is using Go version: devel +40ba4d4e4672 Tue Nov 13 10:45:30 2012
-0800
Are there any known problem with Go not returning memory to the OS on
that version (or the bleeding edge in general)? I do not see memory
problems like this on Linux. Although the nature of my test
environment means I can't do precisely the same test on Linux and
FreeBSD because FreeBSD as it a client site.
On Nov 19, 9:20 pm, John Graham-Cumming <jgrah...@gmail.com> wrote:
> I have a long running Go program that allocates a large amount of
> memory over time. On FreeBSD 64 bit I am seeing a problem where once I
> am done with the memory the RES size as reported by top never
> decreases.
Hmm. I think I might have found the answer to my question in
mem_freebsd.c:
<jgrah...@gmail.com> wrote:
> On Nov 19, 9:20 pm, John Graham-Cumming <jgrah...@gmail.com> wrote:
>> I have a long running Go program that allocates a large amount of
>> memory over time. On FreeBSD 64 bit I am seeing a problem where once I
>> am done with the memory the RES size as reported by top never
>> decreases.
> Hmm. I think I might have found the answer to my question in
> mem_freebsd.c:
On Monday, November 19, 2012 11:51:04 PM UTC, John Graham-Cumming wrote:
> On Nov 19, 11:47 pm, Dave Cheney <d...@cheney.net> wrote: > > I was just chasing that down. You are correct that implementing this > > method should activate the scavenger. I'll prepare a CL.
> It looks like for xBSD (where x is Free, Net and Open) runtime.madvise > is not implemented at all.
I made an experimental patch to Go on FreeBSD amd64 by adding the following to sys_freebsd_amd64.s:
TEXT runtime?madvise(SB),7,$0
MOVQ 8(SP), DI
MOVQ 16(SP), SI
MOVQ 24(SP), DX
MOVQ $75, AX // madvise
SYSCALL
CMPQ AX, $0xfffffffffffff001
JLS 2(PC)
MOVL $0xf1, 0xf1 // crash
RET
And changing mem_freebsd.c so that it actually gets called. That worked.
On Tue, Nov 20, 2012 at 2:40 PM, John Graham-Cumming <jgrah...@gmail.com> wrote:
> On Monday, November 19, 2012 11:51:04 PM UTC, John Graham-Cumming wrote:
>> On Nov 19, 11:47 pm, Dave Cheney <d...@cheney.net> wrote:
>> > I was just chasing that down. You are correct that implementing this
>> > method should activate the scavenger. I'll prepare a CL.
>> It looks like for xBSD (where x is Free, Net and Open) runtime.madvise
>> is not implemented at all.
> I made an experimental patch to Go on FreeBSD amd64 by adding the following
> to sys_freebsd_amd64.s:
> TEXT runtime?madvise(SB),7,$0
> MOVQ 8(SP), DI
> MOVQ 16(SP), SI
> MOVQ 24(SP), DX
> MOVQ $75, AX // madvise
> SYSCALL
> CMPQ AX, $0xfffffffffffff001
> JLS 2(PC)
> MOVL $0xf1, 0xf1 // crash
> RET
> And changing mem_freebsd.c so that it actually gets called. That worked.
2012/11/19 John Graham-Cumming <jgrah...@gmail.com>:
> On Tuesday, November 20, 2012 3:46:37 AM UTC, Dave Cheney wrote:
>> Yup, that is all that is needed. Do you want to propose a CL ?
> What's needed is a CL for all the BSDs and both 32 and 64 bit. I don't have
> all those systems to test against.
Just for reference (in case this isn't what you've done) I'm pretty
sure that MADV_FREE is what we want on FreeBSD. (I don't know about
OpenBSD or NetBSD). MADV_DONTNEED doesn't mean the same thing on Linux
and FreeBSD.
> 2012/11/19 John Graham-Cumming <jgrah...@gmail.com>:
>> On Tuesday, November 20, 2012 3:46:37 AM UTC, Dave Cheney wrote:
>>> Yup, that is all that is needed. Do you want to propose a CL ?
>> What's needed is a CL for all the BSDs and both 32 and 64 bit. I don't have
>> all those systems to test against.
> Just for reference (in case this isn't what you've done) I'm pretty
> sure that MADV_FREE is what we want on FreeBSD. (I don't know about
> OpenBSD or NetBSD). MADV_DONTNEED doesn't mean the same thing on Linux
> and FreeBSD.
----- Original Message ----- From: John Graham-Cumming To: golang-nuts@googlegroups.com Sent: Tuesday, November 20, 2012 3:40 AM
Subject: [go-nuts] Re: Memory use problem on FreeBSD
On Monday, November 19, 2012 11:51:04 PM UTC, John Graham-Cumming wrote:
On Nov 19, 11:47 pm, Dave Cheney <d...@cheney.net> wrote: > I was just chasing that down. You are correct that implementing this > method should activate the scavenger. I'll prepare a CL.
It looks like for xBSD (where x is Free, Net and Open) runtime.madvise is not implemented at all.
I made an experimental patch to Go on FreeBSD amd64 by adding the following to sys_freebsd_amd64.s:
TEXT runtime?madvise(SB),7,$0
MOVQ 8(SP), DI
MOVQ 16(SP), SI MOVQ 24(SP), DX MOVQ $75, AX // madvise SYSCALL
CMPQ AX, $0xfffffffffffff001
JLS 2(PC)
MOVL $0xf1, 0xf1 // crash
RET
And changing mem_freebsd.c so that it actually gets called. That worked.
John.
--
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
Are you planning a CL to fix this problem? My experiments with FreeBSD indicate that MADV_FREE is the way to go, but I am not enough of an expert on the BSD family to do a CL and feel confident that I have fully understood the solution.
> Are you planning a CL to fix this problem? My experiments with FreeBSD
> indicate that MADV_FREE is the way to go, but I am not enough of an expert
> on the BSD family to do a CL and feel confident that I have fully understood
> the solution.
I think Dave was going to let you do it. We can review and make
suggestions if you'd like to continue, otherwise I'm sure Dave can
continue?
Yup. That was the plan, but if you would rather have me do it, please let
me know.
On 21 Nov 2012 07:03, "Devon H. O'Dell" <devon.od...@gmail.com> wrote:
> > Are you planning a CL to fix this problem? My experiments with FreeBSD
> > indicate that MADV_FREE is the way to go, but I am not enough of an
> expert
> > on the BSD family to do a CL and feel confident that I have fully
> understood
> > the solution.
> I think Dave was going to let you do it. We can review and make
> suggestions if you'd like to continue, otherwise I'm sure Dave can
> continue?
