环境
centos7.9(Anolis OS release 7.9) cuda_12.2
问题
输入nvidia-smi时报错误 NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. 
解决方法
先检查驱动是否安装
nvcc -V
centos安装dkms
yum install -y epel-release
yum install -y kernel-headers kernel-devel dkms
ubuntu就直接安装
apt-get install dkms
查看版本
ls /usr/src/|grep nvidia
重新安装驱动
dkms install -m nvidia -v 535.54.03