现象描述

输入nvidia-smi显示如下错误：

jiang@jiang-ThinkStation-P520:~$ nvidia-smiNVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

前几天测试的时候还好好的，突然不行了。

然后查看cuda和cudnn都是有的。

jiang@jiang-ThinkStation-P520:~$ nvcc -Vnvcc: NVIDIA (R) Cuda compiler driverCopyright (c) - NVIDIA CorporationBuilt on Wed_Oct_23_19:24:38_PDT_Cuda compilation tools, release 10.2, V10.2.89jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2#define CUDNN_MAJOR 7#define CUDNN_MINOR 6#define CUDNN_PATCHLEVEL 5--#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)#include "driver_types.h"jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$

原因分析

然后百度发现，有的说是内核自动升级了与英伟达显卡不匹配导致的，得指定内核版本。

解决办法

然后我用了下面方式最后正常了，

（１）首先，查看自己安装的nvidia版本

ls /usr/src | grep nvidia

jiang@jiang-ThinkStation-P520:~$ ls /usr/src | grep nvidianvidia-460.56jiang@jiang-ThinkStation-P520:~$

（２）然后，终端执行一下命令

sudo apt install dkmssudo dkms install -m nvidia -v 460.56

（３）再次输入nvidia-smi，显示：

过程日志

jiang@jiang-ThinkStation-P520:~$ ls /usr/src | grep nvidianvidia-460.56jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ sudo apt install dkms正在读取软件包列表... 完成正在分析软件包的依赖关系树 正在读取状态信息... 完成 dkms 已经是最新版 (2.3-3ubuntu9.7)。dkms 已设置为手动安装。下列软件包是自动安装的并且现在不需要了：libatomic1:i386 libbsd0:i386 libdrm-amdgpu1:i386 libdrm-intel1:i386 libdrm-nouveau2:i386 libdrm-radeon1:i386 libdrm2:i386 libedit2:i386 libelf1:i386 libexpat1:i386libffi6:i386 libfwup1 libgl1:i386 libgl1-mesa-dri:i386 libglapi-mesa:i386 libglvnd0:i386 libglx-mesa0:i386 libglx0:i386 libllvm10:i386 libllvm9 libnvidia-cfg1-440-serverlibnvidia-cfg1-450-server libnvidia-common-440 libnvidia-common-450 libnvidia-common-460 libnvidia-compute-440-server libnvidia-compute-450-serverlibnvidia-decode-440-server libnvidia-decode-450-server libnvidia-encode-440-server libnvidia-encode-450-server libnvidia-extra-440-server libnvidia-extra-450-serverlibnvidia-fbc1-440-server libnvidia-fbc1-450-server libpciaccess0:i386 libsensors4:i386 libstdc++6:i386 libx11-6:i386 libx11-xcb1:i386 libxau6:i386 libxcb-dri2-0:i386libxcb-dri3-0:i386 libxcb-glx0:i386 libxcb-present0:i386 libxcb-sync1:i386 libxcb1:i386 libxdamage1:i386 libxdmcp6:i386 libxext6:i386 libxfixes3:i386 libxnvctrl0libxshmfence1:i386 libxxf86vm1:i386 linux-hwe-5.4-headers-5.4.0-47 linux-hwe-5.4-headers-5.4.0-48 nvidia-compute-utils-440-server nvidia-compute-utils-450-servernvidia-prime nvidia-settings nvidia-utils-440-server nvidia-utils-450-server screen-resolution-extra xserver-xorg-video-nvidia-440-serverxserver-xorg-video-nvidia-450-server使用'sudo apt autoremove'来卸载它(它们)。升级了 0 个软件包，新安装了 0 个软件包，要卸载 0 个软件包，有 73 个软件包未被升级。jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ sudo dkms install -m nvidia -v 460.56Creating symlink /var/lib/dkms/nvidia/460.56/source ->/usr/src/nvidia-460.56DKMS: add completed.Kernel preparation unnecessary for this kernel. Skipping...Building module:cleaning build area...'make' -j8 NV_EXCLUDE_BUILD_MODULES='' KERNEL_UNAME=5.4.0-91-generic IGNORE_CC_MISMATCH='' modules..........Signing module:- /var/lib/dkms/nvidia/460.56/5.4.0-91-generic/x86_64/module/nvidia-uvm.ko- /var/lib/dkms/nvidia/460.56/5.4.0-91-generic/x86_64/module/nvidia-modeset.ko- /var/lib/dkms/nvidia/460.56/5.4.0-91-generic/x86_64/module/nvidia-drm.ko- /var/lib/dkms/nvidia/460.56/5.4.0-91-generic/x86_64/module/nvidia.koSecure Boot not enabled on this system.cleaning build area...DKMS: build completed.nvidia.ko:Running module version sanity check.- Original module- No original module exists within this kernel- Installation- Installing to /lib/modules/5.4.0-91-generic/updates/dkms/nvidia-uvm.ko:Running module version sanity check.- Original module- No original module exists within this kernel- Installation- Installing to /lib/modules/5.4.0-91-generic/updates/dkms/nvidia-modeset.ko:Running module version sanity check.- Original module- No original module exists within this kernel- Installation- Installing to /lib/modules/5.4.0-91-generic/updates/dkms/nvidia-drm.ko:Running module version sanity check.- Original module- No original module exists within this kernel- Installation- Installing to /lib/modules/5.4.0-91-generic/updates/dkms/depmod....DKMS: install completed.jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$ nvidia-smiFri Dec 31 15:52:30 +-----------------------------------------------------------------------------+| NVIDIA-SMI 450.156.00 Driver Version: 460.56 CUDA Version: 11.2||-------------------------------+----------------------+----------------------+| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. || | |MIG M. ||===============================+======================+======================|| 0 GeForce RTX 2080 Off | 00000000:65:00.0 Off | N/A || 22% 50C P0 29W / 225W |0MiB / 7974MiB |0%Default || | | N/A |+-------------------------------+----------------------+----------------------++-----------------------------------------------------------------------------+| Processes:|| GPU GI CI PID Type Process name GPU Memory || ID ID Usage||=============================================================================|| No running processes found |+-----------------------------------------------------------------------------+jiang@jiang-ThinkStation-P520:~$ jiang@jiang-ThinkStation-P520:~$

参考：

1、/p/6b998ba2c6a6

2、/sinat_23619409/article/details/85220561