Installation of deal.II with CUDA

253 views
Skip to first unread message

Chih-Che Chueh

unread,
Nov 7, 2018, 3:15:25 AM11/7/18
to dea...@googlegroups.com
Hi all,

I am trying installing dealII with CUDA in a GPG machine by following this: https://www.dealii.org/9.0.0/external-libs/cuda.html

But I get this error in terms of matrix_tools.cc.o. Is this because I don't turn off Trilinos stuff while cmaking deal.II for successful installation with CUDA?

/usr/bin/cmake3 \
 -DCMAKE_INSTALL_PREFIX=/work1/chueh5851a/software/dealii \
 -DP4EST_DIR=/work1/chueh5851a/software/p4est-2.0/FAST \
 -DTRILINOS_DIR=/work1/chueh5851a/software/Trilinos \
 -D DEAL_II_WITH_MPI:BOOL=ON \
 -D DEAL_II_WITH_THREADS:BOOL=ON \
 -D DEAL_II_WITH_UMFPACK:BOOL=ON \
 -D DEAL_II_WITH_TRILINOS:BOOL=ON \
 -D DEAL_II_WITH_CUDA=ON   \
 -D DEAL_II_WITH_CXX14=OFF \
 -D DEAL_II_WITH_PETSC:BOOL=OFF \
 -D DEAL_II_WITH_SLEPC:BOOL=OFF \
 `pwd`/..

===========make Errors============================================
...
[ 56%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/matrix_tools.cc.o
[ 56%] Building CXX object source/lac/CMakeFiles/obj_lac_debug.dir/matrix_out.cc.o
/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(146): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(154): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(169): error: namespace "Kokkos" has no member "atomic_fetch_or"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(237): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(245): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(260): error: namespace "Kokkos" has no member "atomic_fetch_or"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(304): error: namespace "Kokkos" has no member "atomic_fetch_and"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(314): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(338): error: namespace "Kokkos" has no member "atomic_fetch_or"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_ConcurrentBitset.hpp(348): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_TaskQueue.hpp(226): error: namespace "Kokkos" has no member "atomic_exchange"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_HostBarrier.hpp(110): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_HostBarrier.hpp(113): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_HostBarrier.hpp(130): error: namespace "Kokkos" has no member "atomic_fetch_sub"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_HostBarrier.hpp(131): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/impl/Kokkos_HostBarrier.hpp(239): error: namespace "Kokkos" has no member "atomic_fetch_add"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(93): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(94): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(98): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(102): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(113): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(120): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(112): error: a class or namespace qualified name is required
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(112): error: global-scope qualifier (leading "::") is not allowed
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(112): error: expected a ";"
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(116): error: identifier "base_memory_space" is undefined
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_Concepts.hpp(259): error: static assertion failed with "template arguments must be memory spaces"
          detected during:
            instantiation of class "Kokkos::Impl::MemorySpaceAccess<DstMemorySpace, SrcMemorySpace> [with DstMemorySpace=Kokkos::HostSpace, SrcMemorySpace=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(116): here
            instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(120): error: identifier "base_memory_space" is undefined
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(346): error: identifier "base_memory_space" is undefined
          detected during instantiation of class "Kokkos::MemoryPool<DeviceType> [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): error: name followed by "::" must be a class or namespace name

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(128): error: identifier "CUDA_SAFE_CALL" is undefined

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(132): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(137): error: namespace "Kokkos" has no member "Cuda"

/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(139): error: identifier "CUDA_SAFE_CALL" is undefined

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/set/aux_/item.hpp(47): warning: "char (&operator||(const boost::mpl::s_item<T, Base> &, boost::mpl::aux::type_wrapper<T> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/set/aux_/item.hpp(48): warning: "boost::mpl::aux::no_tag operator%(const boost::mpl::s_item<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/set/aux_/item.hpp(63): warning: "boost::mpl::aux::yes_tag operator%(const boost::mpl::s_mask<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/set/aux_/item.hpp(77): warning: "boost::mpl::aux::no_tag operator%(const boost::mpl::s_unmask<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(486): error: class "Kokkos::Impl::SharedAllocationRecord<<error-type>, void>" has no member "allocate"
          detected during instantiation of "Kokkos::MemoryPool<DeviceType>::MemoryPool(const Kokkos::MemoryPool<DeviceType>::base_memory_space &, size_t, size_t, size_t, size_t) [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(526): error: identifier "base_memory_space" is undefined
          detected during instantiation of "Kokkos::MemoryPool<DeviceType>::MemoryPool(const Kokkos::MemoryPool<DeviceType>::base_memory_space &, size_t, size_t, size_t, size_t) [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_Core_fwd.hpp(242): error: a class or namespace qualified name is required
          detected during:
            processing of template argument list for "Kokkos::Impl::DeepCopy" based on template arguments <<error-type>, Kokkos::HostSpace>
/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(526): here
            instantiation of "Kokkos::MemoryPool<DeviceType>::MemoryPool(const Kokkos::MemoryPool<DeviceType>::base_memory_space &, size_t, size_t, size_t, size_t) [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_Core_fwd.hpp(242): error: expected a ","
          detected during:
            processing of template argument list for "Kokkos::Impl::DeepCopy" based on template arguments <<error-type>, Kokkos::HostSpace>
/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(526): here
            instantiation of "Kokkos::MemoryPool<DeviceType>::MemoryPool(const Kokkos::MemoryPool<DeviceType>::base_memory_space &, size_t, size_t, size_t, size_t) [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

/work1/chueh5851a/software/Trilinos/include/Kokkos_MemoryPool.hpp(526): error: incomplete type is not allowed
          detected during instantiation of "Kokkos::MemoryPool<DeviceType>::MemoryPool(const Kokkos::MemoryPool<DeviceType>::base_memory_space &, size_t, size_t, size_t, size_t) [with DeviceType=<error-type>]"
/work1/chueh5851a/software/Trilinos/include/Sacado_DynamicArrayTraits.hpp(122): here

39 errors detected in the compilation of "/tmp/tmpxft_000389ff_00000000-6_cuda_matrix_free.cpp1.ii".
make[2]: *** [source/matrix_free/CMakeFiles/obj_matrix_free_debug.dir/cuda_matrix_free.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Sincerely,

Chih-Che

Bruno Turcksin

unread,
Nov 7, 2018, 8:29:38 AM11/7/18
to deal.II User Group
Chih-Che


On Wednesday, November 7, 2018 at 3:15:25 AM UTC-5, Chih-Che Chueh wrote:
Hi all,

I am trying installing dealII with CUDA in a GPG machine by following this: https://www.dealii.org/9.0.0/external-libs/cuda.html

But I get this error in terms of matrix_tools.cc.o. Is this because I don't turn off Trilinos stuff while cmaking deal.II for successful installation with CUDA?
This is a known problem when CUDA and Trilinos are both enabled at the same time. If you need both Trilinos and CUDA, you need to use master and make sure that Trilinos is compiled without Sacado support.

Best,

Bruno

Chih-Che Chueh

unread,
Nov 7, 2018, 6:40:24 PM11/7/18
to dea...@googlegroups.com
Hi Bruno,

When I install deal.II with CUDA, I do turn off Trilinos completely by typing this:

/usr/bin/cmake3 \
 -DCMAKE_INSTALL_PREFIX=/work1/chueh5851a/software/dealii \
 -DP4EST_DIR=/work1/chueh5851a/software/p4est-2.0/FAST \
 -D DEAL_II_WITH_MPI:BOOL=ON \
 -D DEAL_II_WITH_THREADS:BOOL=ON \
 -D DEAL_II_WITH_UMFPACK:BOOL=ON \
 -D DEAL_II_WITH_TRILINOS:BOOL=OFF \
 -D DEAL_II_WITH_CUDA=ON   \
 -D DEAL_II_WITH_CXX14=OFF \
 -D DEAL_II_WITH_PETSC:BOOL=OFF \
 -D DEAL_II_WITH_SLEPC:BOOL=OFF \
 `pwd`/..

But I still get a lot of errors coming from somewhere else, as shown below:


[ 65%] Building CXX object source/meshworker/CMakeFiles/obj_meshworker_debug.dir/mesh_worker_vector_selector.cc.o
[ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer.cc.o
[ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst2.cc.o
[ 65%] Building CUDA object source/lac/CMakeFiles/obj_lac_debug.dir/cuda_precondition.cu.o
[ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst3.cc.o
[ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst4.cc.o
[ 65%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_series_fourier.cc.o
/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign

make[1]: *** [source/matrix_free/CMakeFiles/obj_matrix_free_debug.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 65%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_series_legendre.cc.o
/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign

[ 65%] Building CUDA object source/lac/CMakeFiles/obj_lac_debug.dir/cuda_solver_direct.cu.o
[ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/vector_tools_integrate_difference.cc.o
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = float]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1815:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = float]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1815:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<float*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {float*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = float]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1815:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<float*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<float*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {float*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = float]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1815:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<float*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = double]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1816:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = double]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1816:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<double*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {double*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Elements ...>&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = double]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1816:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<double*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<double*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {double*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Elements ...>&&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1293:54:   required from ‘void dealii::CUDAWrappers::PreconditionIC<Number>::initialize(const dealii::CUDAWrappers::SparseMatrix<Number>&, const dealii::CUDAWrappers::PreconditionIC<Number>::AdditionalData&) [with Number = double]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_precondition.cu:1816:16:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<double*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign

make[2]: *** [source/lac/CMakeFiles/obj_lac_debug.dir/cuda_precondition.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 65%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_system.cc.o
[ 67%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/vector_tools_interpolate.cc.o
[ 67%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/vector_tools_point_value.cc.o
/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign

/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:29:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), float*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:29:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<float*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<float*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Args1 ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {float*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Args1 ...>&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:29:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<float*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<float*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Args1 ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {float*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), float*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), float*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Args1 ...>&&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:29:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<float*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {float*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:248:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:51:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:483:67: error: mismatched argument pack lengths while expanding ‘std::is_constructible<_Elements, _UElements&&>’
       return __and_<is_constructible<_Elements, _UElements&&>...>::value;
                                                                   ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:484:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_MoveConstructibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:626:362:   required by substitution of ‘template<class ... _UElements, typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(_UElements&& ...) [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; typename std::enable_if<(((std::_TC<(sizeof... (_UElements) == 1), double*, int*, int*, cusparseMatDescr*>::_NotSameTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>()) && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && (4ul >= 1)), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:51:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:489:65: error: mismatched argument pack lengths while expanding ‘std::is_convertible<_UElements&&, _Elements>’
       return __and_<is_convertible<_UElements&&, _Elements>...>::value;
                                                                 ^~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:490:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_ImplicitlyMoveConvertibleTuple() [with _UElements = {std::tuple<double*, int*, int*, cusparseMatDescr*>}; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<double*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:662:419:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Args1 ...>&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(const std::tuple<_Args1 ...>&) [with _UElements = {double*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<const tuple<_Args1 ...>&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:51:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = const std::tuple<double*, int*, int*, cusparseMatDescr*>&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple: In instantiation of ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<double*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’:
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:686:422:   required by substitution of ‘template<class ... _UElements, class _Dummy, typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Args1 ...>&&>()), bool>::type <anonymous> > constexpr std::tuple< <template-parameter-1-1> >::tuple(std::tuple<_Args1 ...>&&) [with _UElements = {double*, int*, int*, cusparseMatDescr*}; _Dummy = void; typename std::enable_if<((std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_MoveConstructibleTuple<_UElements ...>() && std::_TC<(1ul == sizeof... (_UElements)), double*, int*, int*, cusparseMatDescr*>::_ImplicitlyMoveConvertibleTuple<_UElements ...>()) && std::_TC<(std::is_same<_Dummy, void>::value && (1ul == 1)), double*, int*, int*, cusparseMatDescr*>::_NonNestedTuple<tuple<_Args1 ...>&&>()), bool>::type <anonymous> = <missing>]’
/work1/chueh5851a/CUDA/dealii/source/lac/cuda_solver_direct.cu:51:51:   required from here
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:495:244: error: wrong number of template arguments (5, should be 2)
       return  __and_<__not_<is_same<tuple<_Elements...>,
                                                                                                                                                                                                                                                    ^
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/type_traits:1558:8: note: provided for ‘template<class _From, class _To> struct std::is_convertible’
     struct is_convertible
        ^~~~~~~~~~~~~~
/cm/local/apps/gcc/6.3.0/include/c++/6.3.0/tuple:502:1: error: body of constexpr function ‘static constexpr bool std::_TC<<anonymous>, _Elements>::_NonNestedTuple() [with _SrcTuple = std::tuple<double*, int*, int*, cusparseMatDescr*>&&; bool <anonymous> = true; _Elements = {double*, int*, int*, cusparseMatDescr*}]’ not a return-statement
     }
 ^
make[2]: *** [source/lac/CMakeFiles/obj_lac_debug.dir/cuda_solver_direct.cu.o] Error 1
make[1]: *** [source/lac/CMakeFiles/obj_lac_debug.dir/all] Error 2
[ 67%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/vector_tools_project.cc.o
[ 67%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_enriched.cc.o
[ 67%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_tools.cc.o


--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Bruno Turcksin

unread,
Nov 8, 2018, 8:46:07 AM11/8/18
to dea...@googlegroups.com
Chih-Che

Le mer. 7 nov. 2018 à 18:40, Chih-Che Chueh <chu...@gmail.com> a écrit :
> [ 65%] Building CXX object source/meshworker/CMakeFiles/obj_meshworker_debug.dir/mesh_worker_vector_selector.cc.o
> [ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer.cc.o
> [ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst2.cc.o
> [ 65%] Building CUDA object source/lac/CMakeFiles/obj_lac_debug.dir/cuda_precondition.cu.o
> [ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst3.cc.o
> [ 65%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/solution_transfer_inst4.cc.o
> [ 65%] Building CXX object source/fe/CMakeFiles/obj_fe_debug.dir/fe_series_fourier.cc.o
> /work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used
>
> /work1/chueh5851a/CUDA/dealii/bundled/boost-1.62.0/include/boost/mpl/print.hpp(64): warning: integer conversion resulted in a change of sign
>
> make[1]: *** [source/matrix_free/CMakeFiles/obj_matrix_free_debug.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
It looks like the first error is before what you showed. Also can you
post the detailed.log file.

Best,

Bruno

Chih-Che Chueh

unread,
Nov 8, 2018, 9:09:49 AM11/8/18
to dea...@googlegroups.com
Bruno,


> make[1]: *** [source/matrix_free/CMakeFiles/obj_matrix_free_debug.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
It looks like the first error is before what you showed. Also can you
post the detailed.log file.


See the attached file for your reference and checking.

Sincerely,

Chih-Che 
detailed.log

Bruno Turcksin

unread,
Nov 8, 2018, 9:29:56 AM11/8/18
to dea...@googlegroups.com
Chih-Che
Le jeu. 8 nov. 2018 à 09:09, Chih-Che Chueh <chu...@gmail.com> a écrit :
> See the attached file for your reference and checking.
The testsuite only runs for CUDA 8.0 and 9.0 can you try one of this
version of CUDA and see if that helps?

Best,

Bruno

Chih-Che Chueh

unread,
Nov 8, 2018, 9:49:57 AM11/8/18
to dea...@googlegroups.com
Bruno



> See the attached file for your reference and checking.
The testsuite only runs for CUDA 8.0 and 9.0 can you try one of this
version of CUDA and see if that helps?

When I am using cuda/8.0.61, I get the following error after I do cmake.

[chueh5851a@glogin1 build]$ module list
Currently Loaded Modulefiles:
  1) gcc/6.3.0                        3) blas/gcc/64/3.7.0                5) scalapack/openmpi/gcc/64/2.0.2
  2) mpi/openmpi-3.0.0/gcc630         4) lapack/gcc/64/3.7.0              6) cuda/8.0.61

 -- Include /work1/chueh5851a/CUDA/dealii/cmake/configure/configure_1_cuda.cmake
-- Found CUDA: /pkg/cuda/8.0.61 (found version "8.0")
-- Configured to use CUDA installation at /pkg/cuda/8.0.61
--   CUDA_VERSION: 8.0
--   CUDA_LIBRARIES: /pkg/cuda/8.0.61/lib64/libcudart.so;/pkg/cuda/8.0.61/lib64/libcusparse.so;/pkg/cuda/8.0.61/lib64/libcusolver.so
--   CUDA_INCLUDE_DIRS: /pkg/cuda/8.0.61/include
--   CUDA_USER_INCLUDE_DIRS: /pkg/cuda/8.0.61/include
-- Found CUDA
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP_Fortran: -fopenmp (found version "4.0")
-- Found OpenMP: TRUE (found version "4.5")
-- The CUDA compiler identification is unknown
-- Check for working CUDA compiler: /pkg/cuda/8.0.61/bin/nvcc
-- Check for working CUDA compiler: /pkg/cuda/8.0.61/bin/nvcc -- broken
CMake Error at /usr/share/cmake3/Modules/CMakeTestCUDACompiler.cmake:46 (message):
  The CUDA compiler

    "/pkg/cuda/8.0.61/bin/nvcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp

    Run Build Command:"/usr/bin/gmake" "cmTC_67f8a/fast"
    /usr/bin/gmake -f CMakeFiles/cmTC_67f8a.dir/build.make CMakeFiles/cmTC_67f8a.dir/build
    gmake[1]: Entering directory `/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp'
    Building CUDA object CMakeFiles/cmTC_67f8a.dir/main.cu.o
    /pkg/cuda/8.0.61/bin/nvcc     -x cu -c /work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_67f8a.dir/main.cu.o
    nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
    In file included from /pkg/cuda/8.0.61/bin/..//include/cuda_runtime.h:78:0,
                     from <command-line>:0:
    /pkg/cuda/8.0.61/bin/..//include/host_config.h:119:2: error: #error -- unsupported GNU version! gcc versions later than 5 are not supported!
     #error -- unsupported GNU version! gcc versions later than 5 are not supported!
      ^~~~~
    gmake[1]: *** [CMakeFiles/cmTC_67f8a.dir/main.cu.o] Error 1
    gmake[1]: Leaving directory `/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp'
    gmake: *** [cmTC_67f8a/fast] Error 2




  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  cmake/configure/configure_1_cuda.cmake:124 (ENABLE_LANGUAGE)
  build/CMakeFiles/CMakeTmp/evaluate_expression.tmp:1 (FEATURE_CUDA_CONFIGURE_EXTERNAL)
  cmake/macros/macro_evaluate_expression.cmake:30 (INCLUDE)
  cmake/macros/macro_configure_feature.cmake:244 (EVALUATE_EXPRESSION)
  cmake/configure/configure_1_cuda.cmake:164 (CONFIGURE_FEATURE)
  cmake/macros/macro_verbose_include.cmake:19 (INCLUDE)
  CMakeLists.txt:124 (VERBOSE_INCLUDE)


-- Configuring incomplete, errors occurred!
See also "/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeOutput.log".
See also "/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeError.log".

Sincerely,

Chih-Che

Bruno Turcksin

unread,
Nov 8, 2018, 9:58:49 AM11/8/18
to dea...@googlegroups.com
Chih-Che,

Le jeu. 8 nov. 2018 à 09:49, Chih-Che Chueh <chu...@gmail.com> a écrit :
> -- The CUDA compiler identification is unknown
> -- Check for working CUDA compiler: /pkg/cuda/8.0.61/bin/nvcc
> -- Check for working CUDA compiler: /pkg/cuda/8.0.61/bin/nvcc -- broken
> CMake Error at /usr/share/cmake3/Modules/CMakeTestCUDACompiler.cmake:46 (message):
>   The CUDA compiler
>
>     "/pkg/cuda/8.0.61/bin/nvcc"
>
>   is not able to compile a simple test program.
>
>   It fails with the following output:
>
>     Change Dir: /work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp
>
>     Run Build Command:"/usr/bin/gmake" "cmTC_67f8a/fast"
>     /usr/bin/gmake -f CMakeFiles/cmTC_67f8a.dir/build.make CMakeFiles/cmTC_67f8a.dir/build
>     gmake[1]: Entering directory `/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp'
>     Building CUDA object CMakeFiles/cmTC_67f8a.dir/main.cu.o
>     /pkg/cuda/8.0.61/bin/nvcc     -x cu -c /work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_67f8a.dir/main.cu.o
>     nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
>     In file included from /pkg/cuda/8.0.61/bin/..//include/cuda_runtime.h:78:0,
>                      from <command-line>:0:
>     /pkg/cuda/8.0.61/bin/..//include/host_config.h:119:2: error: #error -- unsupported GNU version! gcc versions later than 5 are not supported!
>      #error -- unsupported GNU version! gcc versions later than 5 are not supported!
>       ^~~~~
>     gmake[1]: *** [CMakeFiles/cmTC_67f8a.dir/main.cu.o] Error 1
>     gmake[1]: Leaving directory `/work1/chueh5851a/CUDA/dealii/build/CMakeFiles/CMakeTmp'
>     gmake: *** [cmTC_67f8a/fast] Error 2
This is because of a change we just did here. You will need to use GCC 5 instead of GCC 6.

Best,

Bruno

Chih-Che Chueh

unread,
Nov 13, 2018, 7:24:27 AM11/13/18
to dea...@googlegroups.com
Bruno,

You are right. With installing gcc 5 and module load it, deal.II can be successfully installed with CUDA in the GPU machine I am using. Thanks!

Right now, I would like to know this: is there any existing deal.II code example I could use to run on a machine with CUDA-aware MPI?

Thanks!  

Sincerely,

Chih-Che


--

Daniel Arndt

unread,
Nov 13, 2018, 7:32:05 AM11/13/18
to deal.II User Group
Chih-Che,

 
Right now, I would like to know this: is there any existing deal.II code example I could use to run on a machine with CUDA-aware MPI?
 
Using MPI and CUDA together is pretty much work in progress. On the developer branch, LinearAlgebra::distributed::Vector supports
storing its elements on a CUDA device, but the values are copied to the CPU for MPI communication.
The pull request https://github.com/dealii/dealii/pull/7303 allows to use CUDA-aware MPI instead but the memory transfer needs to be optimized.

Best,
Daniel

Chih-Che Chueh

unread,
Nov 18, 2018, 7:33:25 AM11/18/18
to dea...@googlegroups.com
Daniel,


Using MPI and CUDA together is pretty much work in progress. On the developer branch, LinearAlgebra::distributed::Vector supports
storing its elements on a CUDA device, but the values are copied to the CPU for MPI communication.
The pull request https://github.com/dealii/dealii/pull/7303 allows to use CUDA-aware MPI instead but the memory transfer needs to be optimized.

If I could get involved in this development, I would be helpful to optimize the memory transfer between GPU and CPU by testing different CUDA C programming schemes like this: https://devblogs.nvidia.com/how-overlap-data-transfers-cuda-cc/

In fact, as a principal investigator at Taiwan, I am qualified to submit jobs into Taiwania, a recently established super computing machine established in 2017, equipped with NVIDIA Tesla P100 SXM2. Moreover, sometime next year, we will have another super computing machine available to use (Taiwania 2), which now ranks 20th in the world with NVIDIA Tesla V100 SXM2.


Sincerely,

Chih-Che


 

Bruno Turcksin

unread,
Nov 19, 2018, 5:40:25 PM11/19/18
to dea...@googlegroups.com
Chih-Che,

Le dim. 18 nov. 2018 à 07:33, Chih-Che Chueh <chu...@gmail.com> a écrit :
> If I could get involved in this development, I would be helpful to optimize the memory transfer between GPU and CPU by testing different CUDA C programming schemes like this: https://devblogs.nvidia.com/how-overlap-data-transfers-cuda-cc/
That would be nice. Playing with streams is not something that we have
time to do in the foreseeable future. So if you are interested in
doing it, that would be a great!.

Best,

Bruno

Chih-Che Chueh

unread,
Nov 19, 2018, 6:47:59 PM11/19/18
to dea...@googlegroups.com
Hi Bruno,


> If I could get involved in this development, I would be helpful to optimize the memory transfer between GPU and CPU by testing different CUDA C programming schemes like this: https://devblogs.nvidia.com/how-overlap-data-transfers-cuda-cc/
That would be nice. Playing with streams is not something that we have
time to do in the foreseeable future. So if you are interested in
doing it, that would be a great!.


I am interested in testing the overlapping and different schemes developed you and others in deal.II.

I have one question: if I want to install the latest "develop" version, I just simply type this: git clone https://github.com/dealii/dealii, right? If not, which command I need to type?

Thanks!

Sincerely,

Chih-Che
 

Daniel Arndt

unread,
Nov 20, 2018, 7:46:39 AM11/20/18
to deal.II User Group
Hi Chih-Che,

 
I am interested in testing the overlapping and different schemes developed you and others in deal.II.

I have one question: if I want to install the latest "develop" version, I just simply type this: git clone https://github.com/dealii/dealii, right? If not, which command I need to type?
Yes, you can obtain a recent version using that command. However, the PR is not merged yet, so you would need to run something like the following afterwards to checkout the relevant branch:

git checkout -b masterleinad-cuda_aware_mpi master
git pull https://github.com/masterleinad/dealii.git cuda_aware_mpi

Best,
Daniel

Massimo Bernaschi

unread,
Dec 29, 2018, 9:57:18 AM12/29/18
to deal.II User Group
I use this thread since I am having troubles installing dealii with CUDA support.
I am using the "candi.sh" route (hopefully this makes sense). 
I modified the dealii.package (file attached) to turn off Trilinos and to turn on CUDA support. I am using GCC 5.3 and CUDA 9.2. 
The compilation fails with the following errors:
/root/deal.ii-candi/tmp/unpack/deal.II-v9.0.1/include/deal.II/matrix_free/cuda_matrix_free.templates.h(391): error: a variable length array cannot have static storage duration

/root/deal.ii-candi/tmp/unpack/deal.II-v9.0.1/include/deal.II/matrix_free/cuda_matrix_free.templates.h(392): error: a variable length array cannot have static storage duration

I also attach the full candi_build.log file. Could the problem be that I am also forcing the usage of the bundled Boost? 
Did someone succeed in compiling dealII with CUDA using the candi.sh script? 
Thanks in advance and best regards,
Massimo
candi_build.log
dealii.package

Bruno Turcksin

unread,
Dec 31, 2018, 8:15:17 AM12/31/18
to dea...@googlegroups.com
Massimo,

If you can use the development version of deal. The CUDA support is
pretty new and we have made a lot of progress since the latest
release. Look here
https://github.com/dealii/candi/blob/dealii-9.0/candi.cfg#L67 to
switch to the development version. If you want to use the release
version, I would advise you to switch to CUDA 8.0

Best,

Bruno

Le sam. 29 déc. 2018 à 09:57, Massimo Bernaschi
<massimo....@gmail.com> a écrit :
> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/xHmid5d5-iI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.

Massimo Bernaschi

unread,
Jan 1, 2019, 6:40:43 PM1/1/19
to dea...@googlegroups.com
Thanks Bruno, in the end I succeeded in installing the latest stable version from github. I recall briefly what I did in
the hope that it can help others.
2)cmake -DCMAKE_INSTALL_PREFIX=/usr/local/ /root/dealii
3)cmake DEAL_II_WITH_CUDA:BOOL=ON
4)make
5)make install
However when I tried to compile the solver_03.cu in tests/cuda I had few undefined symbols (due to the cxx11 prefix issue described in other parts).
What I did was to compile the following "dummy.c" file
_ZN6dealii18deal_II_exceptions9internals25get_cusolver_error_stringB5cxx11E16cusolverStatus_t() { 
     ZN6dealii18deal_II_exceptions9internals25get_cusolver_error_stringE16cuso}
_ZN6dealii18deal_II_exceptions9internals25get_cusparse_error_stringB5cxx11E16cusparseStatus_t() { 
   _ZN6dealii18deal_II_exceptions9internals25get_cusparse_error_stringE16cusp}
_ZN6dealii9LogStream10get_streamB5cxx11Ev() { return 0; }

Then I link the file with the following command:
nvcc -o solver_03 solver_03.o -L/usr/local/lib/ -ldeal_II -std c++11 /root/dummy.o
(before I compiled the source with
nvcc -c solver_03.cu -O3 -I/usr/local/include -I ../../bundled/boost-1.62.0/include -std c++11).
(I don't know if the 
The program works correctly on the GPU (I double checked running it under the control of the nvprof command).
This is with CUDA 9.2.
Thanks again and Best Wishes to everyone for a Happy 2019,
Massimo


You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--
--- Massimo Bernaschi: Istituto Applicazioni del Calcolo ----
|  IAC-CNR                  | e-mail: massimo....@cnr.it |
|  Via dei Taurini, 19     | phone: +39 06 49937350              |
|  00185 Roma - ITALY | fax:   +39 06 4404306                  |
------------------------------------------------------------------------------------------------------------------------
|Skype nickname: m.bernaschi                                                                           |
------------------------------------------------------------------------------------------------------------------------
|See http://www.iac.cnr.it/~massimo for my GPG public key or check                       |
|GnuPG Public Key Fingerprint (keyserver.linux.it)                                                    |
|pub  1024/CAA3FB48 2001/01/04 Massimo Bernaschi <mas...@iac.rm.cnr.it>  |
|     Key fingerprint = 3EFF 7AFF F8A4 F34E 382B  DD81 57F3 700A CAA3 FB48 |
------------------------------------------------------------------------------------------------------------------------
Reply all
Reply to author
Forward
0 new messages