Unsupported gpu architecture 'compute_86'

759 views
Skip to first unread message

Arkadi

unread,
Feb 17, 2022, 5:41:54 AM2/17/22
to kaldi-help
Hello,

I try to compile kaldi on my new pc. Specs are :
Model name: Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz ( X86_64 )
OS: Ubuntu 20.04.3 LTS 
GPU : NVIDIA RTX A5000
GPU-DRIVERS: NVIDIA-SMI 510.39.01    Driver Version: 510.39.01    CUDA Version: 11.6

Attached the output of deviceQuery script. I understand that my GPU is using sm_86.
I've added to Kaldi.mk "-gencode arch=compute_86,code=sm_86"

Still while compiling the "src" folder, I'm getting the error "Unsupported gpu architecture 'compute_86' ".
I'm using the latest kaldi source code from github.

Tried all I found in this group and on Google search but without success.
Any advice ?

Thanks
Screen Shot 2022-02-17 at 12.38.26.png

Daniel Povey

unread,
Feb 17, 2022, 5:43:33 AM2/17/22
to kaldi-help
You have to make clean and recompile after modifying the Makefile; make does not track dependencies on the Makefile.

--
Go to http://kaldi-asr.org/forums.html to find out how to join the kaldi-help group
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/03be96b2-2079-4a4d-960e-e3bb5fa543dfn%40googlegroups.com.

Arkadi

unread,
Feb 17, 2022, 6:07:56 AM2/17/22
to kaldi-help
I looked on the Makefile from the src folder as well on the Makefile from cudamatrix folder.
Didn't find anything match to different cuda archs

What did you mean ?

Jan Yenda Trmal

unread,
Feb 17, 2022, 6:11:22 AM2/17/22
to kaldi-help
I think you might be using nvcc from an older cuda, like pre-11.1 or something.
y.

Arkadi

unread,
Feb 17, 2022, 6:19:26 AM2/17/22
to kaldi-help
My CUDA version ( as nvidia-smi displays ) is 11.6 
nvcc - release 10.1, V10.1.243

Jan Yenda Trmal

unread,
Feb 17, 2022, 6:20:55 AM2/17/22
to kaldi-help
I think that confirms that -- your sdk is 10.1 
y.

Arkadi

unread,
Feb 17, 2022, 6:26:37 AM2/17/22
to kaldi-help
Sorry but It's not clear to me.
I downloaded the latest version from nvidia download site . 
It mentioned there that the version is 11, while the nvcc is included in the installation package.
Also my cuda version is 11.6 . 
do you mean I need to upgrade my nvcc ? because I didn't find how to do it  separately ..
Can you help with that ?

Jan Yenda Trmal

unread,
Feb 17, 2022, 6:33:56 AM2/17/22
to kaldi-help
it's a too generic problem to be able to help. perhaps you installed cuda using the system package manager or just didn't  uninstall the previous one, so you have two conflicting  versions now installed?
What does the following command outputs on the command line?
which nvcc


Arkadi

unread,
Feb 17, 2022, 6:41:54 AM2/17/22
to kaldi-help
which nvcc :
 /usr/bin/nvcc

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

nvcc --version : 
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

nvidia-smi: 
+-----------------------------------------------------------------------------+

| NVIDIA-SMI 510.39.01    Driver Version: 510.39.01    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A5000    On   | 00000000:2D:00.0 Off |                  Off |
| 30%   30C    P8    17W / 230W |      1MiB / 24564MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found   

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


./deviceQuery :
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "NVIDIA RTX A5000"
  CUDA Driver Version / Runtime Version          11.6 / 11.6
  CUDA Capability Major/Minor version number:    8.6
  Total amount of global memory:                 24253 MBytes (25431310336 bytes)
  (64) Multiprocessors, (128) CUDA Cores/MP:     8192 CUDA Cores
  GPU Max Clock rate:                            1695 MHz (1.70 GHz)
  Memory Clock rate:                             8001 Mhz
  Memory Bus Width:                              384-bit
  L2 Cache Size:                                 6291456 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  1536
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Compute Preemption:            Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 45 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.6, CUDA Runtime Version = 11.6, NumDevs = 1, Device0 = NVIDIA RTX A5000
Result = PASS


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


Any more info needed ?
                                           

Justin Luitjens

unread,
Feb 17, 2022, 8:44:41 AM2/17/22
to kaldi...@googlegroups.com
Why are you using 11.6 driver and 10.1 toolkit?  Can you upgrade your toolkit to 11.6 too?

Arkadi

unread,
Feb 17, 2022, 8:50:01 AM2/17/22
to kaldi-help
I just followed the instuctions from nvidia website.
Probabaly this is not correct.
Do you have the tutorial for installing it correctly ? 

Justin Luitjens

unread,
Feb 17, 2022, 9:29:02 AM2/17/22
to kaldi...@googlegroups.com
Where did you install the 11.6 toolkit?  Where does which nvcc show nvcc is from?  You probably have multiple toolkits installed and are using the wrong one.

Arkadi

unread,
Feb 17, 2022, 10:34:24 AM2/17/22
to kaldi-help
I just followed the link from  here ( nvidia official ) . Installed it on a new ubuntu20


Here is the commands I ran in the terminal :

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          48
On-line CPU(s) list:             0-47
Thread(s) per core:              2
Core(s) per socket:              12
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85

Model name:                      Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz
Stepping:                        7
CPU MHz:                         1000.587
CPU max MHz:                     3200.0000
CPU min MHz:                     1000.0000
BogoMIPS:                        4400.00
Virtualization:                  VT-x
L1d cache:                       768 KiB
L1i cache:                       768 KiB
L2 cache:                        24 MiB
L3 cache:                        33 MiB
NUMA node0 CPU(s):               0-11,24-35
NUMA node1 CPU(s):               12-23,36-47
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; TSX disabled
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtsc
                                 p lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est
                                  tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowpref
                                 etch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad
                                  fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw
                                 avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp
                                 _pkg_req pku ospke avx512_vnni md_clear flush_l1d arch_capabilities


lsb_release -a
No LSB modules are available.
Distributor ID:        Ubuntu
Description:        Ubuntu 20.04.3 LTS
Release:        20.04
Codename:        focal

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


Really don't know what is the issue...

Justin Luitjens

unread,
Feb 17, 2022, 11:07:29 PM2/17/22
to kaldi...@googlegroups.com
It's almost certainly that you have the wrong compiler in your path.  You probably have multiple toolkits installed.

Usually nvcc is found at /usr/local/cuda not /usr/bin.

Perhaps someone has symlinked in /usr/bin

Your nvcc --version should say 11.6 not 10.1

Arkadi Gurevich

unread,
Feb 18, 2022, 8:26:58 AM2/18/22
to kaldi...@googlegroups.com
Yes ! This was the issue indeed.
Thanks,
Arkadi

Sent from my iPhone

On 18 Feb 2022, at 6:07, Justin Luitjens <luit...@gmail.com> wrote:


You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/ILFC5FJlo60/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CALCo63e4aG_2_nOJTP%2BH3MXuMe%2BR66%2BofeLX%3DZawXjJfCKT%3DqA%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages