Page 1 of 1

makefile.include for intel compilers uses multithreaded mkl

PostPosted: Thu Jun 09, 2016 5:06 pm
by fish
To everyone building VASP with the Intel compilers and MKL libraries,

The makefile.include provided with the vasp.5.4.1 distribution uses the "-mkl" option for the linker (or the FCL variable). Note that this will link to the multithreaded MKL libraries under v16 of the Inter compilers. This can lead to poor performance, if the number of openMP threads generated by multithreaded MKL is greater than the number of cores on your node.

I would recommend using the -mkl=sequential option or not using the -mkl option and explicitly listing the required MKL libraries in the BLAS variable of makefile.include.

I have attached an makefile.include with this change as an example.

John J. Low
Argonne National Laboratory

Re: makefile.include for intel compilers uses multithreaded

PostPosted: Mon Mar 27, 2017 12:50 am
by tgomez
Hi, could you upload the file sample?
I have problems with the compilation and your input would help me so much.
Thanks

Re: makefile.include for intel compilers uses multithreaded

PostPosted: Fri Sep 22, 2017 7:27 pm
by cacarden
Dear John:
It seems that you did not attach the make.iluclude. Would you please share with us?
Thanks,
Carlos

Re: makefile.include for intel compilers uses multithreaded

PostPosted: Thu May 10, 2018 3:39 pm
by fish
Since I could not add an attachment, I cut and pasted an example of a "makefile.include". Sorry for the delay.
# Precompiler options
CPP_OPTIONS= -DHOST=\"Bebop\ BDW\ IFC17.0.4\ IMPI17.0.3\"\
-DMPI -DMPI_BLOCK=8000 \
-Duse_collective \
-DscaLAPACK \
-DCACHE_SIZE=4000 \
-Davoidalloc \
-Duse_bse_te \
-Dtbdyn \
-Duse_shmem

CPP = fpp -f_com=no -free -w0 $*$(FUFFIX) $*$(SUFFIX) $(CPP_OPTIONS)

FC = mpiifort
FCL = mpiifort -mkl=sequential -lstdc++

FREE = -free -names lowercase

FFLAGS = -assume byterecl -w
OFLAG = -O2 -fma -xCORE-AVX2
OFLAG_IN = $(OFLAG)
DEBUG = -O0

MKL_PATH = $(MKLROOT)/lib/intel64
BLAS =
LAPACK =
BLACS = -lmkl_blacs_intelmpi_lp64
SCALAPACK = $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS)

OBJECTS = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o

INCS =-I$(MKLROOT)/include/fftw

LLIBS = $(SCALAPACK) $(LAPACK) $(BLAS)


OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o

# For what used to be vasp.5.lib
CPP_LIB = $(CPP)
FC_LIB = $(FC)
CC_LIB = icc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o

# For the parser library
CXX_PARS = icpc

LIBS += parser
LLIBS += -Lparser -lparser -lstdc++

# Normally no need to change this
SRCDIR = ../../src
BINDIR = ../../bin

#================================================
# GPU Stuff

CPP_GPU = -DCUDA_GPU -DRPROMU_CPROJ_OVERLAP -DUSE_PINNED_MEMORY -DCUFFT_MIN=28 -UscaLAPACK

OBJECTS_GPU = fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d_gpu.o fftmpiw_gpu.o

CC = icc
CXX = icpc
CFLAGS = -fPIC -DADD_ -Wall -openmp -DMAGMA_WITH_MKL -DMAGMA_SETAFFINITY -DGPUSHMEM=300 -DHAVE_CUBLAS

CUDA_ROOT ?= /usr/local/cuda/
NVCC := $(CUDA_ROOT)/bin/nvcc -ccbin=icc
CUDA_LIB := -L$(CUDA_ROOT)/lib64 -lnvToolsExt -lcudart -lcuda -lcufft -lcublas

GENCODE_ARCH := -gencode=arch=compute_30,code=\"sm_30,compute_30\" \
-gencode=arch=compute_35,code=\"sm_35,compute_35\" \
-gencode=arch=compute_60,code=\"sm_60,compute_60\"

MPI_INC = $(I_MPI_ROOT)/include64/