Intel MKL allows fast math computation on CPU and NCCL enables fast multiple GPU communication. Both of them are desirable for the running experiments, so let's get them.
Install Intel MKL
I will prefer to install all the packages in
$HOME/apps. Thus, first create the directory.
Get the download link from Intel MKL's page, and download on the server: https://software.intel.com/en-us/mkl .
cd ~/apps # Put your real link here. wget http://registrationcenter-download.intel.com/akdlm/irc_nas/tec/xxx/l_mkl_xxx.tgz tar xzvf l_mkl*.tgz rm l_mkl_*.tgz cd l_mkl_* ./install.sh
Clone torch library and run the installation script.
git clone https://github.com/torch/distro.git ~/apps/torch --recursive cd ~/apps/torch bash install-deps ./install.sh
Error: no default constructor exists for class ….
If you get this error, try to get the latest cuda driver. Make sure the driver version is not a pre-release version.
Clone the NCCL repository and compile the project.
cd ~/apps git clone https://github.com/NVIDIA/nccl cd nccl make CUDA_HOME=/usr/local/cuda test sudo make install # Copy libnccl files to cuda's folder, # so you don't have to modify the environment paths sudo cp /usr/local/lib/libnccl* /usr/local/cuda/lib64/
Restart the terminal to make the torch avaiable
If you are running tmux, try
source ~/.zshrc or
It turns out that Torch does not yet support cudnn 6.0 currently.