Object is in wrong state, VecAssembly

[Karol Lewandowski]

Hi,

Recently after struggling with error (pasted below) when trying to use SNES solver, I have realized that I don't fully understand what do petsc functions like VecAssemblyBegin/End() actually do and at which point we should use them. I'd be grateful for a brief explanation.

Cheers,

[0]PETSC ERROR: MoFEM version 0.5.99

[0]PETSC ERROR: MoFEM git commit id d9e9e3e2581a41f38c6990a4514c515a39081498

[0]PETSC ERROR: See http://mofem.eng.gla.ac.uk/mofem/html/guidelines_bug_reporting.html for bug reporting.

[0]PETSC ERROR: See http://mofem.eng.gla.ac.uk/mofem/html/faq_and_bugs.html for trouble shooting.

[0;39m[0;49m[1;31m[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------

[0;39m[0;49m[0]PETSC ERROR: Object is in wrong state

[0]PETSC ERROR: You cannot call this after you have called VecSetValues() but

 before you have called VecAssemblyBegin/End()

[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.

[0]PETSC ERROR: Petsc Release Version 3.8.0, unknown 

[0]PETSC ERROR: /Users/karollewandowski/moFEM/users_modules_debug/basic_finite_elements/elasticity_New/elasticity_New on a arch-darwin-c-opt named Karols-MBP by karollewandowski Sat Nov 11 00:21:22 2017

[0]PETSC ERROR: Configure options --with-debugging=0 --download-fblaslapack=1 --download-superlu_dist=1 --download-metis=1 --download-parmetis=1 --download-hypre=1 --download-mumps=1 --download-scalapack=1 --download-blacs=1 --download-moab=1 --download-hdf5=1 --download-netcdf=/Users/karollewandowski/moFEM//petsc/netcdf-4.3.3.1.tar.gz --download-mumps=1 --download-openmpi=1

[0]PETSC ERROR: #1 VecSet() line 545 in /Users/karollewandowski/moFEM/petsc/src/vec/vec/interface/rvector.c

[0]PETSC ERROR: #2 VecZeroEntries() line 1201 in /Users/karollewandowski/moFEM/petsc/src/vec/vec/interface/vector.c

[0]PETSC ERROR: #3 SnesRhs() line 67 in /Users/karollewandowski/moFEM/mofem-cephas/mofem/src/petsc/impl/SnesCtx.cpp

[0]PETSC ERROR: #4 SNESComputeFunction() line 2195 in /Users/karollewandowski/moFEM/petsc/src/snes/interface/snes.c

[0]PETSC ERROR: #5 SNESSolve_NEWTONLS() line 175 in /Users/karollewandowski/moFEM/petsc/src/snes/impls/ls/ls.c

[0]PETSC ERROR: #6 SNESSolve() line 4106 in /Users/karollewandowski/moFEM/petsc/src/snes/interface/snes.c

 

[Lukasz Kaczmraczyk]

Hello,

PETSc can manage different type of vectors in particular MPI (distributed) vectors. The user has to be consistent on all processors wether it is going to insert or add values to vector, since the operation of adding or inserting is at the end collective. Collective means that all processors do the same thing, i.e. adding or inserting. So one can not do adding and inserting at same time, it has to be some communication in between those two types of operations.  Inserting and adding is not commutative and since you do calculations you can not guarantee order of operations parallel processing. 

So for example if one first like insert and then add values, a assembly operation has to be called in between stages;

// Inserting (stage 1)
VecSet(x,value);
VecSetValues(x,nb,indices,values,INSERT_VALUES);
VecAssemblyBegin(Vx);
VecAssemblyEnd(x);

// Adding (stage 2)
VecSetValues(x,nb,indices,values,ADD_VALUES);
VecAssemblyBegin(x);
VecAssemblyEnd(x);

Note that if you call VecAssemblyBegin/End() you have communication between processors. And this is very slow!!! You should have algorithm which call this function as little as possible, optimally once for load/time step.

You should also look at http://www.mcs.anl.gov/petsc/petsc-current/docs/manual.pdf chapter 2.

In your particular case you like to zero entries in the vector, before that operation you need to assemble vector. Error suggest that you have some general  problem with sequence of operations. It is not important if you run single or multi processor problem, algorithm has to be the same for both cases.  

Lukasz