Memory not freed up when deleting JuMP objects

[Andrew B. Martin]

Hello,

I'm try to generate a very large model by writing it piece by piece. After I write a piece, I want to discard it so that my system's memory resources don't get exhausted. 

The problem I'm having is that memory isn't freed up when I discard a piece.

# System memory 500/15039MB

julia> d = buidModel();
"success"

# System memory at 8883/15039MB

# delete the model object - corresponds to about a 1 Gb LP file
# I observe the same effect if I enter
# julia> d["m"] = 0;

julia> delete!(d, “m”);

# System memory unchanged

julia> gc()

# System memory unchanged

# delete a list of affine expressions used to make constraints in the model - estimated 1GB - 2GB RAM

julia> delete!(d, “aff_exprs”);

# System memory unchanged

julia> gc()

# System memory unchanged

julia> d = 0

# System memory unchanged

julia> gc()

# System memory 8255/15039MB

The dictionary d should contain all the data that takes up almost 9GB of RAM. Instead when I delete it only a fraction of that is freed up.

After deleting d, when I type whos() I get a list of functions and modules that together shouldn't take up nearly 8GB of RAM.

Why isn't all the memory freed when I delete d or the values of d?

Another way of asking this is could there be hidden copies of objects taking up space that I can’t identify with whos()?

I'm using Ubuntu 14.04.1 LTS with Julia 0.3.8 and JuMP 0.9.1. I also observe this behaviour on Windows 7 with Julia 0.3.10 and JuMP 0.9.2.

[Joey Huchette]

Are you sure the dictionary isn’t still holding on to a reference to your large objects?

julia> d = Dict([1,2],[3,4])
Dict{Int64,Int64} with 2 entries:
  2 => 4
  1 => 3

julia> names(d)
6-element Array{Symbol,1}:
 :slots
 :keys
 :vals
 :ndel
 :count
 :deleter

julia> d.vals
16-element Array{Int64,1}:
 0
 0
 0
 0
 0
 4
 0
 0
 0
 0
 0
 0
 0
 0
 0
 3

julia> delete!(d, 1)
Dict{Int64,Int64} with 1 entry:
  2 => 4

julia> d.vals
16-element Array{Int64,1}:
 0
 0
 0
 0
 0
 4 
 0
 0
 0
 0
 0
 0
 0
 0
 0
 3 # I thought I deleted you!

​

--
You received this message because you are subscribed to the Google Groups "julia-opt" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-opt+...@googlegroups.com.
Visit this group at http://groups.google.com/group/julia-opt.
For more options, visit https://groups.google.com/d/optout.

[Andrew B. Martin]

Thanks, Joey! Sounds like it could be the answer.

I'll check back tomorrow with whether that's what's happening.

If the reference is still there, how do you get rid of it?

[Joey Huchette]

I’m not entirely sure (there’s a d.deleter(d) function that seems like it does the trick, but you don’t want to mess around with the internals). I’d ask on the julia-users mailing list.

​

--

[Andrew B. Martin]

Thanks.

I've asked the question on julia-users: https://groups.google.com/d/msg/julia-users/3FyBRMNPNG4/5B1HEpS3_JsJ

Upon checking it doesn't look like anything lingers in d.vals after I delete! a dictionary value, or reassign it to 0.