-ksp_monitor

[blackbird]

Dear all,

When comparing serial and parallel solution of a problem modelled with my userelement I get the message
  NO CONVERGENCE REASON:  Iterations exceeded 
While the serial run shows no problems at all it is getting even stranger, because the solver just continues after this message and my load-displacement curve of serial and parallel problem are similar.

The parfeap call gets options -ksp_type cg -pc_type jacobi. By adding also -ksp_monitor I get an output like

N KSP Residual norm XXX

with N seems to be integer, XXX is real. N is counting from zero, XXX is diminishing each line. N is set to zero each time feap starts a new newton iteration or advances in time. Could you tell me what is N?

Also each time N exceeds 10000, i get the error message but the calculation will continue and also i got quite good results compared with serial solution. While "no convergence" is not intended (as not shown in serial solution) i guess it is a problem with the parallel feap (or some option i should set rather than the one i did)?

[FEAP_Admin]

The "N KSP Residual norm XXX" is a message from PETSc.  N is the iteration and XXX is (I think) the preconditioned residual (not the true residual).  In parFEAP we test in parfeap/usolve.F for the reason that the solver exited with a call to KSPGetConvergedReason(  ).  In your case, you have exceeded the number of maximum (default) allowed iteration for the iterative solver.  N is the iteration counter for PETSc's Krylov solver.  Note that FEAP will continue even if you have not converged.  This can be quite dangerous and results need to be carefully examined.

Your results are looking ok to you, which means that you are apparently close enough to converged to move on in your computation.  Note that the serial solver is performing a direct solve (by default) which is completely different.  You could ask the serial code to also perform an iterative solve and you should see similar behavior to the parallel solve.

[blackbird]

is there any way to set the tolerances and maximum number of iterations number for the method via command line arguments? Is there any documentation about feap's command line arguments incombination with petsc?

[blackbird]

there seems to be no documentation provided by petsc itself but i found something helpful posted by petsc user. command line arguments should be

-ksp_rtol ## - setting relative tolerance
-ksp_atol ## - setting absolute tolerance
-ksp_divtol ## - setting diverging tolerance
-ksp_max_it ## - setting maximum number of iterations

while -ksp_max_it works fine, the other 3 options have no influence on the solver, which can be seen in the "-ksp_view" output:

KSP Object: 8 MPI processes
  type: cg
  maximum iterations=10, initial guess is zero
  tolerances:  relative=1e-08, absolute=1e-16, divergence=1e+16
  left preconditioning
  using PRECONDITIONED norm type for convergence test

no matter which (rtol,atol,divtol) option i use - the values shown here stay the same every time... do you know a way to change the tolerance values?

[FEAP_Admin]

Look in parfeap/usolve.F for

Code: [Select]

          call KSPGetTolerances (kspsol,p_rtol,p_atol,p_dtol,p_maxit,
     &                           ierr)
          if(pmaxit.le.0) then
            pmaxit = p_maxit
          endif
          call KSPSetTolerances (kspsol,  itol,  atol,  dtol,pmaxit,
     &                           ierr)                   ! From command data
We only utilize the maximum iterations parameter from the command line.  The rest are ignored in favor of the ones set using FEAP's TOLerance command (see pp. 488-490 of the user manual).  If you want to use the command line options, you can pass the values read by KSPGetTolerances into KSPSetTolerances but make sure to add some logic so that the FEAP values are used if you have not set them on the command line.

btw, section 4.3.2 of the PETSc manual discusses such options.  Others are also discussed in the PETSc manual.

[luc]

On a slightly different tack: you might want to use something stronger than jacobi as your preconditioner to get faster convergence.
You can also use PETSc to perform LU as it is done in serial FEAP by passing the command

-ksp_type preonly -pc_type lu    [pass -pc_factor_mat_solver_package <mumps,pastix,superlu_dist> if you do a parallel LU, obviously you need to have these packages installed first]