Query using hdf5 with deal.ii

[RAJAT ARORA]

Hello all,

I am using deal.ii to solve a 3D solid mechanics problem. I am using Petsc and P4est along with deal.ii

There is a previous post related to my query but it does not talk in detail about using hdf5.

 

My code currently follows step 40 and outputs one vtu file from each processor and one pvtu file from a rank 0 processor.

However, as the number of processors grow, this is generating huge number of files as output which inturn takes a lot of time to copy / move to my desktop for postprocessing.

Also, if I output data from multiple equations (being solved concurrently), the mesh data is replicated in each of these files.

I came across using hdf5 which might solve this issue. Since there is no example that demonstrates the use of hdf5, I have a a question before 

I start exploring and using it. Will this output just 1 file independent of the number of processors on which the code is run ? 

This is a source file of aspect that demonstrates using hdf5 in detail.

Also, for snapshot creation and restart purposes, I output 1 file from each processor containing gauss point data at a current time step. This is also not scalable for the same reasons.

Is there any way that I can get only 1 file (or maximum 1 file per node) that contains the data from all the gauss points spread over multiple processors ? Or can this data be included while 

doing triangulation.save() which saves triangulation and data that is associated with it using Parallel solution transfer class ?

Thanks for the help.

-Rajat Arora

PhD. Student

Carnegie Mellon University

[Wolfgang Bangerth]

Rajat,

> I am using deal.ii to solve a 3D solid mechanics problem. I am using
> Petsc and P4est along with deal.ii
>
> There is a previous post

> <https://groups.google.com/forum/#!searchin/dealii/hdf5%7Csort:relevance/dealii/_51WLg4jDGI/yvFDxSLeTswJ> related

> to my query but it does not talk in detail about using hdf5.
>
> My code currently follows step 40 and outputs one vtu file from each
> processor and one pvtu file from a rank 0 processor.
> However, as the number of processors grow, this is generating huge
> number of files as output which inturn takes a lot of time to copy /
> move to my desktop for postprocessing.
> Also, if I output data from multiple equations (being solved
> concurrently), the mesh data is replicated in each of these files.
>
> I came across using hdf5 which might solve this issue. Since there is no
> example that demonstrates the use of hdf5, I have a a question before
> I start exploring and using it. Will this output just 1 file independent
> of the number of processors on which the code is run ?
>
> This is asource file of aspect

> <https://github.com/geodynamics/aspect/blob/6db2fa985d24f7d4e60dc8cde9b9a2d63817113b/source/particle/output/hdf5.cc>

> that demonstrates using hdf5 in detail.

This is the only example I know of.

But you may want to look at the VTU part there again: it allows writing
the data from whole *groups* of processors into one file so that you
don't end up with one file per processor. We typically use this for
larger simulations.

> Also, for snapshot creation and restart purposes, I output 1 file from
> each processor containing gauss point data at a current time step. This
> is also not scalable for the same reasons.
> Is there any way that I can get only 1 file (or maximum 1 file per node)
> that contains the data from all the gauss points spread over multiple
> processors ? Or can this data be included while
> doing triangulation.save() which saves triangulation and data that is
> associated with it using Parallel solution transfer class ?

Both is possible. You can attach the data to the triangulation. ASPECT
again does this with particle data, for example -- I don't know where
exactly this happens, but you can take a look at the particles directory.

It is also possible to let each processor collect its own data, put
things into one long array, send that to processor 0, and let that
processor put it into a central file. I believe that that, too, is what
ASPECT does. You may want to look into the
source/simulator/checkpoint_restart.cc file. (I'm going to say that
we've implemented these pieces so long ago that I don't recall all of
the details. So I may be wrong in my claims above, but that's where I
would look things up.)

Best
W.



--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/