dump.gsd error : "Too many files open"

[Surendra Walter Singaram]

Hello everyone,
    
I'm using HOOMD 2.0.1 and wondering if anyone knows how to deal with the
following dump.gsd error:

**ERROR**: dump.gsd: Too many open files - ./last_frame.gsd
Traceback (most recent call last):
  File "./../wales_diamonds.py", line 371, in <module>
  File "/share/apps/scisoft/GNU-4.9.3-compilations/HOOMD/2.0.1-CUDA7.5_SINGLE_NOMPI/hoomd/dump.py", line 555, in __init__
RuntimeError: Error opening GSD file


 I use dump.gsd to create a restart file by overwriting an exiting gsd file called "last_frame.gsd". 
My dump.gsd is part of a loop and gets called every 40,000 steps.  I issue the command:

gsdframe = dump.gsd(filename='./last_frame.gsd',period=None,group=group.all(),overwrite=True,truncate=True,static=[])

If I call dump.gsd less frequently (more precisely 5x less frequently) and keep the total simulation time the same the error,
as expected, disappears.

I hope you can help me.

Best wishes,
     Walter

[Michael Howard]

I'm not positive, but I think that is an error from your filesystem indicating that you have too many file handles open. Assuming you aren't opening any files elsewhere, probably what is happening is that however you are using the dump command in your script is causing the gsd file handles to be repeatedly opened but never closed.

In general, doing this sort of thing to a filesystem is discouraged, especially if it is on a cluster using lustre, and could earn you a ban from your sys admin. If you want to dump a restart file every 40,000 timesteps, you should probably rewrite your script to use period=40000 (outside the loop), which should cause the gsd file handle to only be opened once.

Regards... Mike

[Surendra Walter Singaram]

Hi Mike,
 Thank you for your suggestion.  I suspected I was having an issue with the file handles.
I have monte-carlo moves interspersed with the dynamics, so that at the beginning loop.

I issue:
   context.initialize()
followed by:
    init.read_gsd(filename='./last_frame.gsd')

I didn't think the read_gsd was causing the issue, but maybe it is also playing a role in the error? 
In general, is there a smarter way to handle monte carlo moves that are sampled with
such high frequency? (That is, other than dumping the current frame at the bottom of
the loop and reading it at the beginning).

Thank you,
   Walter 

[Michael Howard]

Hi Walter,

It would depend on the type of MC moves you are doing. If you are modifying the configuration or topology in a way that can be done using python, then I would probably try to use the snapshot interface (take_snapshot and restore_snapshot), which is reasonably fast access via numpy arrays. If you are doing something else to the system, then you might be able to use the callback option to run().

--Mike

[Joshua Anderson]

I second what Michael suggests on take_ and restore_snapshot. That interface was designed with MC / transition path sampling type techniques in mind. There is no need to take a round trip to disk if you want to capture and restore state within a single python script. And there is no need to continually reinitialize the execution context. You can use the same execution context, pair potentials, integrators, etc..., and simply restore the state.

Regarding writing the last frame to a gsd file, why not do:
``gsdframe = dump.gsd(filename='./last_frame.gsd',period=40000,group=group.all(),overwrite=True,truncate=True,static=[])``?

truncate=True causes dump.gsd to ovewrite frame 0 in the file every period steps which it seems is what you want to do. Of course, this only works if you continue to use the same context.

dump.gsd and init.gsd both internally create class objects that will hold file handles for their lifetime. These objects are not held, so the file handles should be freed. However, the python garbage collector is not very aggressive at freeing unused objects. If you are writing or reading gsd files in a loop, you may need to force garbage collection every loop iteration so that the file handles will be freed.

Also, simulation contexts (and nlist, pair, and other objects) take up a lot of memory. If you continually initialize contexts in a loop, you should ensure that you do not hold onto any references to that context or compute objects and that you force python garbage collection to avoid memory leaks where so many contexts are kept alive in memory.

------
Joshua A. Anderson, Ph.D.
Research Area Specialist, Chemical Engineering, University of Michigan
http://www-personal.umich.edu/~joaander/

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.
To post to this group, send email to hoomd...@googlegroups.com.
Visit this group at https://groups.google.com/group/hoomd-users.
For more options, visit https://groups.google.com/d/optout.

[Surendra Walter Singaram]

Hello Michael and Joshua,

Thank you both for your reply.

In the earlier version of my script I had tried using the take_ and restore_snapshot.  This looked liked an elegant way to proceed for my system.  I gave up using this method because the pair potentials wouldn't update properly resulting in my subunits not interacting.  Briefly, I have 3 types of rigid bodies (call the types A, B, and C) which can interact with each other.   The rigid bodies can switch types (i.e. A -> B, B->C, etc.) during the MC step.  I defined all my pair potentials, nlists, etc. before the take_ and restore_snapshot stage of the script.  While the rigid bodies would switch types, I found that they "forgot" their pair potentials resulting in rigid bodies that no longer interacted.  I naively assumed this was caused by tags changing and HOOMD not updating the nlists properly. 

In a desperate attempt to get things working I tried this context_initialize(), init.read_gsd, dump.gsd strategy, which does work albeit for low sampling frequencies. 

Could you perhaps tell me why I saw such odd behavior in my implementation of take_ and restore_snapshot()?  Thank you.

Best wishes,
      Walter

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.

[Michael Howard]

If all you are doing is changing particle types, then you definitely want to use the snapshot interface. Restoring a snapshot should force the neighbor lists to rebuild, since it is the same as reinitializing the system. Did you make sure that you are changing the types of both the central particle in the rigid body and the constituent particles?

Regards... Mike

[Joshua Anderson]

You need to call create_bodies after a restore_snapshot to update the composite body data structures.

------
Joshua A. Anderson, Ph.D.
Research Area Specialist, Chemical Engineering, University of Michigan
http://www-personal.umich.edu/~joaander/

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.

[Surendra Walter Singaram]

Hi Mike and Joshua,

Thank you so much for your reply!  In my old version I only changed the central particle, deleted all constituent particles, then ran create_bodies.  I also don't think I called create_bodies after a restore_snapshot.  Let me give this a try.

--Walter

[Eric Irrgang]

Presumably the constituent particles are in the snapshot, and the non-default create_bodies(False) argument should be used, right?

I would have thought this was necessary when restoring from the GSD file, too. Does init.gsd take care of that?

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.

[Joshua Anderson]

create_bodies(False) is sufficient if the body specifications do not change. The reporter was not clear, but seemed to indicate that the body types themselves were changing, thus requiring create_bodies(True) to rebuild the necessary constituent particles.

No, initi.read_gsd does not automatically call create_bodies.

------
Joshua A. Anderson, Ph.D.
Research Area Specialist, Chemical Engineering, University of Michigan
http://www-personal.umich.edu/~joaander/

[Jens Glaser]

You can also use dump.gsd() with the argument of group=group.rigid_center(). In that case you would only store the central particles,

and call create_bodies(True) to recreate the constituent particle after restoring from the gsd.

Jens

[Surendra Walter Singaram]

Thank you everyone for your comments.

To be clear, I've always used the take_ and restore_snapshot interface. I just found that I get the best performance when I loop over dump.gsd(filename='./last_frame.gsd') followed by context.initialize() and init.read_gsd(filename='./last_frame.gsd').

 My simulation has different types of rigid bodies as well as a linear polymer, so Jens last suggestion didn't work because HOOMD complained about being
unable to find bonds.

I've tried going back to my earlier version, which did not include the dump.gsd/context.initialize()/init.read_gsd.  The old issue of the rigid bodies not interacting seems to have resurfaced.
Here's the place that I think is causing the issue.  I don't understand why the constituent particles of rigid bodies are forgetting their pair potentials.  Looking forward to hearing your ideas.

beginning of loop

    #AFTER MONTE CARLO STEP

    #update membership
    groupC0.force_update()  #rigid body A central particles groupC0=group.type(name='C0-particles',type='C0',update=True)
    groupC1.force_update()  #rigid body B central particles groupC1=group.type(name='C1-particles',type='C1',update=True)
    groupC2.force_update()  #rigid body C central particles groupC1=group.type(name='C2-particles',type='C2',update=True)
    #delete constituent particles
    #hoomd gets mad otherwise
    for t in Consttags: #these are the tags of the constituent particles
        system.particles.remove(t)
    rigid.create_bodies(create=True)
#a few lines later I issue:
    centralrigid = group.rigid_center()
    nonrigid = group.nonrigid()
    integrableparts = group.union(name='int-parts-lang',a=centralrigid, b=nonrigid)
    imode.set_params(dt=ts)
    langevin = md.integrate.langevin(group=integrableparts, kT=1.0, seed=seed, dscale=1.0)
    langevin.disable()
    del langevin
end of loop

[Eric Irrgang]

When you took the snapshot, did you set bonds=True?

If your rigid body definitions change between dumps and restorations, don't dump all of the particles (e.g. group.union, group.non_rigid() and group.rigid_center in gsd.dump) and use create_bodies(create=True). If you can reuse the constituent particles between restorations, use group.all and create_bodies(create=False)

When you say the constituent particles are forgetting their pair potentials, do you mean that you create the particles and they don't have the type you specified in the rigid body you just defined? Or that they have the type specified in the rigid body definition and a pair interaction defined for that type, but it isn't being applied? Or that you alter the definitions of the pair potentials in the system and want a way to easily revert to an earlier set of pair potentials?

If it is the last, then maybe what you're looking for is an object of the hoomd.md.pair.coeff class.

[Surendra Walter Singaram]

Hi Eric,

I've responded to your comments in purple

On Wednesday, June 14, 2017 at 5:02:28 PM UTC-4, Eric Irrgang wrote:

When you took the snapshot, did you set bonds=True? No, I use the default settings

If your rigid body definitions change between dumps and restorations, don't dump all of the particles (e.g. group.union, group.non_rigid() and group.rigid_center in gsd.dump) and use create_bodies(create=True). Since my rigid bodies can change I always have used create_bodies(create=True).  I also have a linear polymer in my system, so I tried to dump using group.union() of non-rigid and rigid_center.  HOOMD complained upon restoration that it couldn't find any bonds.  When I dump using group,all() I don't have problems with restoration.  If you can reuse the constituent particles between restorations, use group.all and create_bodies(create=False)

When you say the constituent particles are forgetting their pair potentials, do you mean that you create the particles and they don't have the type you specified in the rigid body you just defined? Or that they have the type specified in the rigid body definition and a pair interaction defined for that type, but it isn't being applied? This is what mean, Weird, right? Or that you alter the definitions of the pair potentials in the system and want a way to easily revert to an earlier set of pair potentials?

[Eric Irrgang]

For snapshots, bonds=False, by default.

Are there bonds involving constituent particles?

At this point, it would help if you could post a complete script with a minimal reproducible example.

--
You received this message because you are subscribed to the Google Groups "hoomd-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.

[Surendra Walter Singaram]

Hi Eric,
There are no bonds involving constituent particles.  I hope to post a complete script with a minimal reproducible example soon.
Thank you,
 Walter

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users...@googlegroups.com.

[Surendra Walter Singaram]

Hi Eric,
I'm so sorry for my late reply.

After much experimenting, I've found a simple fix for my problem.  When using rigid bodies, however, one must make sure bonds=True is set upon taking a snapshot; otherwise, HOOMD misplaces the constituent particles. The take_snapshot/restore_snapshot interface work beautifully for MC simulations.  I'm happy to post a working example, if you'd like.  Thank you all for your help.

Best wishes,
Walter

[Eric Irrgang]

Thanks for confirming. You're welcome to post a script you think will be helpful to others.

I just couldn't see from the script you posted that you were providing the necessary override to the default `bonds` parameter, and if that wasn't the problem, I didn't see anything else obvious.

Glad you got it working. :-)

To unsubscribe from this group and stop receiving emails from it, send an email to hoomd-users+unsubscribe@googlegroups.com.