Error durind checkpoint / restart using parallel distributed solution transfer

[RAJAT ARORA]

Hello all,

I am trying to use the checkpoint / restart code with the help of the example given in the documentation here

I am getting a long error while saving a parallel vector and parallel distributed triangulation using the following code.

Code Used:

string triangulation_name = "mesh_restart";

parallel::distributed::SolutionTransfer<dim, LA::MPI::Vector> sol_trans_x(dof_handler);
sol_trans_x.prepare_serialization(locally_owned_x);

triangulation.save(triangulation_name.c_str());


//LA::MPI::Vector corresponds to PETSC MPI Vector

// locally_owned_x is solution vector reinitialized as 
// locally_owned_x.reinit(locally_owned_dofs, MPI_COMM_WORLD);

Error here:

An error occurred in line <1253> of file </home/rajat/Documents/Code-Libraries/deal/dealii8.3/dealii-8.3.0/include/deal.II/lac/petsc_vector_base.h> in function
    void dealii::PETScWrappers::VectorBase::extract_subvector_to(ForwardIterator, ForwardIterator, OutputIterator) const [with ForwardIterator = const unsigned int*; OutputIterator = double*]
The violated condition was:
    index>=static_cast<unsigned int>(begin) && index<static_cast<unsigned int>(end)
The name and call sequence of the exception was:
    ExcInternalError()
Additional Information:
This exception -- which is used in many places in the library -- usually indicates that some condition which the author of the code thought must be satisfied at a certain point in an algorithm, is not fulfilled. An example would be that the first part of an algorithm sorts elements of an array in ascending order, and a second part of the algorithm later encounters an an element that is not larger than the previous one.


There is usually not very much you can do if you encounter such an exception since it indicates an error in deal.II, not in your own program. Try to come up with the smallest possible program that still demonstrates the error and contact the deal.II mailing lists with it to obtain help.


Stacktrace:
-----------
#0  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: void dealii::PETScWrappers::VectorBase::extract_subvector_to<unsigned int const*, double*>(unsigned int const*, unsigned int const*, double*) const
#1  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: void dealii::DoFCellAccessor<dealii::DoFHandler<3, 3>, false>::get_dof_values<dealii::PETScWrappers::MPI::Vector, double*>(dealii::PETScWrappers::MPI::Vector const&, double*, double*) const
#2  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: void dealii::DoFCellAccessor<dealii::DoFHandler<3, 3>, false>::get_interpolated_dof_values<dealii::PETScWrappers::MPI::Vector, double>(dealii::PETScWrappers::MPI::Vector const&, dealii::Vector<double>&, unsigned int) const
#3  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: dealii::parallel::distributed::SolutionTransfer<3, dealii::PETScWrappers::MPI::Vector, dealii::DoFHandler<3, 3> >::pack_callback(dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)
#4  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: std::_Function_handler<void (dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*), std::_Bind<std::_Mem_fn<void (dealii::parallel::distributed::SolutionTransfer<3, dealii::PETScWrappers::MPI::Vector, dealii::DoFHandler<3, 3> >::*)(dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)> (dealii::parallel::distributed::SolutionTransfer<3, dealii::PETScWrappers::MPI::Vector, dealii::DoFHandler<3, 3> >*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>)> >::_M_invoke(std::_Any_data const&, dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)
#5  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: std::function<void (dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)>::operator()(dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*) const
#6  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: std::_Function_handler<void (dealii::TriaIterator<dealii::CellAccessor<3, 3> >, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*), std::function<void (dealii::TriaIterator<dealii::CellAccessor<3, 3> > const&, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)> >::_M_invoke(std::_Any_data const&, dealii::TriaIterator<dealii::CellAccessor<3, 3> >, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)
#7  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: std::function<void (dealii::TriaIterator<dealii::CellAccessor<3, 3> >, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*)>::operator()(dealii::TriaIterator<dealii::CellAccessor<3, 3> >, dealii::parallel::distributed::Triangulation<3, 3>::CellStatus, void*) const
#8  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0:
#9  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: dealii::parallel::distributed::Triangulation<3, 3>::attach_mesh_data()
#10  /home/rajat/Documents/Code-Libraries/deal/dealii8.3/installed-dealii/lib/libdeal_II.g.so.8.3.0: dealii::parallel::distributed::Triangulation<3, 3>::save(char const*) const
#11  ./fdm: FDM::Fdm<3>::create_snapshot()
#12  ./fdm: Problem<3>::run()
#13  ./fdm: main

[RAJAT ARORA]

Hello,

This is because of the vector locally_owned_x.

As per the documentation, during the serialization, the vector should be ghosted. 

If I make the change, the code works fine.

[Jean-Paul Pelteret]

Hi Rajat,

Thanks for posting how you solved your problem! I'm glad to know that you found the issue, and this will be helpful to know. Perhaps we can mention/reinforce it in the documentation.

Regards,

J-P

[RAJAT ARORA]

Hello,

Yes professor. I thought this might be helpful to anyone who gets a similar error and is looking for a solution.

[Jean-Paul Pelteret]

Hi Rajat,

Great, thanks for marking the solution. Unless you want to do so yourself, I'll post an issue on GitHub tomorrow to indicate that the error message could be made more clear. 

Regards,

J-P

P.S. I'm not a Prof., but rather an ordinary PhD. But thanks for the ego boost anyway ;-)