Memory error from utilities.cc

[Raghunandan Pratoori]

Hello team,

I am trying to run a simulation with grid refine factor 7. I know this is significantly large and any improper code will raise memory issues. I am in fact getting memory issues after completion of first time step and I am not able to pin point where I probably am making a mistake. The below is the error message.

periodic facepairs: 1
   Number of active cells: 9437
   Number of degrees of freedom: 8586756 (6440067+2146689)
 Solving for Displacement:
   rhs_norm : 0.0414408    Number of CG iterations: 55
   rhs_norm : 3.98697e-05    Number of CG iterations: 121
CONVERGED!
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
----------------------------------------------------
Exception on processing:

Aborting!
----------------------------------------------------

--------------------------------------------------------
An error occurred in line <1055> of file </home/rnp/Software/dealii-build/tmp/unpack/deal.II-v9.3.0/source/base/utilities.cc> in function
    void dealii::Utilities::System::posix_memalign(void**, std::size_t, std::size_t)
The violated condition was:
    ierr == 0
Additional information:
    Your program tried to allocate some memory but this allocation failed.
    Typically, this either means that you simply do not have enough memory
    in your system, or that you are (erroneously) trying to allocate a
    chunk of memory that is simply beyond all reasonable size, for example
    because the size of the object has been computed incorrectly.

    In the current case, the request was for 134217728 bytes.

I would highly appreciate any pointers regarding this.

Thanks in advance,

Raghunandan.

[Wolfgang Bangerth]

On 8/19/22 13:29, Raghunandan Pratoori wrote:
>
> I am trying to run a simulation with grid refine factor 7. I know this is
> significantly large and any improper code will raise memory issues. I am in
> fact getting memory issues after completion of first time step and I am not
> able to pin point where I probably am making a mistake.

Raghunandan:
somewhere in your code, there is a place where something wants to allocate 134
MB of memory, but your system does not have this much memory left. You need to
find out where this is, either by strategically placing print statements into
your code to see how far it runs, or (better!) by running the program in a
debugger.

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Raghunandan Pratoori]

Thank you for your suggestion Prof. Bangerth. I was able to pinpoint the problem in my code. I am trying to write all the stress and strain components to the output *vtu files. Below is how initialize the variables needed for that before filling them up with quadrature point history. The lines of code in red are where the memory error is arising from. Is there a better way of handling this process, or am I making some fundamental mistake?

//Output stress and strain componenets
        std::vector<std::vector<Vector<double>>> history_field_stress(dim, std::vector<Vector<double>>(dim)),
                                                local_history_values_at_qpoints(dim, std::vector<Vector<double>>(dim)),
                                                local_history_fe_values(dim, std::vector<Vector<double>>(dim));

        std::vector<std::vector< Vector<double>>> history_field_strain(dim, std::vector<Vector<double>>(dim));

        for (unsigned int i=0; i<dim; i++)
            for (unsigned int j=0; j<dim; j++)
            {
                history_field_stress[i][j].reinit(history_dof_handler.n_dofs());
                local_history_values_at_qpoints[i][j].reinit(qf_cell.size());
                local_history_fe_values[i][j].reinit(history_fe.dofs_per_cell);
                history_field_strain[i][j].reinit(history_dof_handler.n_dofs());
            }

Thanks in advance,

Raghunandan.

[Wolfgang Bangerth]

On 8/20/22 12:56, Raghunandan Pratoori wrote:
>
>         for (unsigned int i=0; i<dim; i++)
>             for (unsigned int j=0; j<dim; j++)
>             {
> history_field_stress[i][j].reinit(history_dof_handler.n_dofs());
>
> local_history_values_at_qpoints[i][j].reinit(qf_cell.size());
>
> local_history_fe_values[i][j].reinit(history_fe.dofs_per_cell);
> history_field_strain[i][j].reinit(history_dof_handler.n_dofs());
>             }

This does not look crazy, unless you have a large number of degrees of
freedom. How large is history_dof_handler.n_dofs() in your case?

[Raghunandan Pratoori]

Hello Prof. Bangerth,

I have 2 sets of dof_handlers. The one that is giving me a problem has 6440067 dofs and it becomes 6x when initializing history_field_stress or history_field_strain. I also plan on increasing these in future simulations.

Can this only be addressed by increasing the memory allotted for the simulation? Or is there a way to decrease the memory demand?

Thanks in advance,

Raghunandan.

[Wolfgang Bangerth]

On 8/23/22 07:22, Raghunandan Pratoori wrote:
>
> I have 2 sets of dof_handlers. The one that is giving me a problem has 6440067
> dofs and it becomes 6x when initializing history_field_stress or
> history_field_strain. I also plan on increasing these in future simulations.

Well, at 8 bytes per 'double' variable, you need 6,440,067 * 6 * 8 =
309,123,216 bytes = 309 MB of memory to do what you want to do for just this
one step. If your machine does not have 309 MB left, you are bound to get the
error you describe. There is nothing you can do about this other than (i) get
a machine with more memory, (ii) use distributed computing where the 6M
unknowns are distributed across more than just one machine.

[Raghunandan Pratoori]

Thank you for your reply Prof. Bangerth. I believe I am already using distributed computing in my code. But I am not sure it is true for the post processing part. I use write_vtu and write_pvtu_record to write the output files. Is that the best way to do it in distributed computing? Is there any example where distributed computing is implemented in the post processing so that I can compare my code and see if it can be improved?

Thanks in advance,

Raghunandan.

[Wolfgang Bangerth]

On 8/25/22 16:56, Raghunandan Pratoori wrote:
> Thank you for your reply Prof. Bangerth. I believe I am already using
> distributed computing in my code. But I am not sure it is true for the post
> processing part. I use write_vtu and write_pvtu_record to write the output
> files. Is that the best way to do it in distributed computing? Is there any
> example where distributed computing is implemented in the post processing so
> that I can compare my code and see if it can be improved?

I see now why you have so many entries in your solution vector :-)

I don't think I have anything to point to, but the general approach needs to
be that on every process you only allocate memory for each locally owned DoF
(i.e., locally owned vector entry), rather than for *every* DoF or vector
entry. That is the only way this can scale.

You can easily query how many locally owned DoFs there are on the current
process, using functions in class DoFHandler. You then probably have to
translate between global indices j and the how-many-th locally owned DoF index
j corresponds to. To this end, ask DoFHandler for the IndexSet corresponding
to the locally owned indices; class IndexSet has functions that can translate
in both directions.

I hope this helps!