cuda - Linker errors when compiling OpenMP `target` directive using Clang 6 (trunk) with nvptx target -

i want run openmp4+ code on nvidia gpu using llvm/clang. downloaded , compiled llvm/clang/omp libraries trunk following instructions found here: https://clang.llvm.org/get_started.html , here: https://openmp.llvm.org/. did not build compiler-rt , libcxx, don't think make difference.

my cmake command this: cmake -g "unix makefiles" ../llvm -dcmake_build_type=release -dopenmp_enable_libomptarget=on

i wrote basic program single openmp target directive:

int main(void) {     #pragma omp target     {     }     return 0; } 

and compile this: /home/user/opt/llvm/bin/clang++ -v main.cpp -fopenmp -lomptarget -fopenmp-targets=nvptx64-nvidia-cuda --cuda-path=/home/user/opt/pgi/linux86-64/2017/cuda/8.0

in case ask: yes, haven't set paths compiler made sure ld_library_path points libomptarget located.

and here output/error after executing above command (the last ~10 lines show error):

<if information, go last 10 lines see error> clang version 6.0.0 (trunk 312875) target: x86_64-unknown-linux-gnu thread model: posix installeddir: /home/user/opt/llvm/bin found candidate gcc installation: /usr/lib/gcc/i686-linux-gnu/6 found candidate gcc installation: /usr/lib/gcc/i686-linux-gnu/6.4.0 found candidate gcc installation: /usr/lib/gcc/i686-linux-gnu/7 found candidate gcc installation: /usr/lib/gcc/i686-linux-gnu/7.2.0 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/4.8 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/4.9 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/5 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.1 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/6 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/6.4.0 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/7 found candidate gcc installation: /usr/lib/gcc/x86_64-linux-gnu/7.2.0 selected gcc installation: /usr/lib/gcc/x86_64-linux-gnu/7.2.0 candidate multilib: .;@m64 selected multilib: .;@m64 found cuda installation: /home/user/opt/pgi/linux86-64/2017/cuda/8.0, version 7.0  "/home/user/opt/llvm/bin/clang-6.0" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /home/user/opt/llvm/lib/clang/6.0.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward -internal-isystem /usr/local/include -internal-isystem /home/user/opt/llvm/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /usr/local/include -internal-isystem /home/user/opt/llvm/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 190 -fopenmp -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/main-be2d35.bc -x c++ main.cpp -fopenmp-targets=nvptx64-nvidia-cuda clang -cc1 version 6.0.0 based upon llvm 6.0.0svn default target x86_64-unknown-linux-gnu ignoring nonexistent directory "/include" ignoring nonexistent directory "/include" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward" ignoring duplicate directory "/usr/local/include" ignoring duplicate directory "/home/user/opt/llvm/lib/clang/6.0.0/include" ignoring duplicate directory "/usr/include/x86_64-linux-gnu" ignoring duplicate directory "/usr/include" #include "..." search starts here: #include <...> search starts here:  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward  /usr/local/include  /home/user/opt/llvm/lib/clang/6.0.0/include  /usr/include/x86_64-linux-gnu  /usr/include end of search list.  "/home/user/opt/llvm/bin/clang-6.0" -cc1 -triple nvptx64-nvidia-cuda -aux-triple x86_64-unknown-linux-gnu -s -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fmath-errno -no-integrated-as -fuse-init-array -mlink-cuda-bitcode /home/user/opt/pgi/linux86-64/2017/cuda/8.0/nvvm/libdevice/libdevice.compute_20.10.bc -target-feature +ptx42 -target-cpu sm_20 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /home/user/opt/llvm/lib/clang/6.0.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward -internal-isystem /usr/local/include -internal-isystem /home/user/opt/llvm/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /usr/local/include -internal-isystem /home/user/opt/llvm/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 190 -fopenmp -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/main-7ffbd7.s -x c++ main.cpp -fopenmp-is-device -fopenmp-host-ir-file-path /tmp/main-be2d35.bc clang -cc1 version 6.0.0 based upon llvm 6.0.0svn default target x86_64-unknown-linux-gnu ignoring nonexistent directory "/include" ignoring nonexistent directory "/include" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0" ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward" ignoring duplicate directory "/usr/local/include" ignoring duplicate directory "/home/user/opt/llvm/lib/clang/6.0.0/include" ignoring duplicate directory "/usr/include/x86_64-linux-gnu" ignoring duplicate directory "/usr/include" #include "..." search starts here: #include <...> search starts here:  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0  /usr/lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward  /usr/local/include  /home/user/opt/llvm/lib/clang/6.0.0/include  /usr/include/x86_64-linux-gnu  /usr/include end of search list. "/home/user/opt/pgi/linux86-64/2017/cuda/8.0/bin/ptxas" -m64 -o0 -v --gpu-name sm_20 --output-file /tmp/main-64fc86.cubin /tmp/main-ca9e59.s -c ptxas info    : 1 bytes gmem, 8 bytes cmem[14] ptxas info    : compiling entry function '__omp_offloading_803_18004c0_main_l3' 'sm_20' ptxas info    : function properties __omp_offloading_803_18004c0_main_l3 0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads ptxas info    : used 5 registers, 32 bytes cmem[0] ptxas info    : function properties __omp_offloading_803_18004c0_main_l3_worker 24 bytes stack frame, 8 bytes spill stores, 8 bytes spill loads "/home/user/opt/pgi/linux86-64/2017/cuda/8.0/bin/nvlink" -o /tmp/main-f247e3.out -v -arch sm_20 -l/home/user/opt/llvm/lib -lomptarget-nvptx main-64fc86.cubin nvlink error   : undefined reference '__kmpc_kernel_init' in 'main-64fc86.cubin' nvlink error   : undefined reference '__kmpc_kernel_deinit' in 'main-64fc86.cubin' nvlink error   : undefined reference '__kmpc_kernel_parallel' in 'main-64fc86.cubin' nvlink error   : undefined reference '__kmpc_kernel_end_parallel' in 'main-64fc86.cubin' nvlink info    : 1 bytes gmem, 8 bytes cmem[14] nvlink info    : function properties '__omp_offloading_803_18004c0_main_l3': nvlink info    : used 18 registers, 24 stack, 0 bytes smem, 32 bytes cmem[0], 0 bytes lmem clang-6.0: error: fatbinary command failed exit code 255 (use -v see invocation) 

any idea library supposed have these __kmpc* symbols? tried run:

nm libomptarget.so | grep __kmpc_kernel_parallel

and

nm libomptarget.rtl.cuda.so | grep __kmpc_kernel_parallel,

but both commands return nothing.

finally, if remove -fopenmp-targets=nvptx64-nvidia-cuda compile flags, no linker errors. of course in case no cuda code generated.

any feedback may me figure out going on, these symbols supposed located, , why not there, more welcome.

unfortunately omp target offloading support nvptx device hasn't been upstreamed yet. recent branch available on github build instructions @ wiki https://github.com/clang-ykt/clang/wiki