01_vector_add test compile error. Just want to confirm can this test be well supported.

[FatJhon]

Describe the bug

user guide test vector_add test, just fail to make_so,get error information as follows:
ret = subprocess.check_call(cc_cmd)
File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/rh/gcc-toolset-11/root/usr/bin/gcc', '/tmp/tmp59kzdlsc/kernel.s', '-O3', '-shared', '-fPIC', '-Wno-psabi', '-o', '/tmp/tmp59kzdlsc/kernel.cpython-310-x86_64-linux-gnu.so', '-lm', '-lTritonCPURuntime', '-lsleef', '-lgcc', '-L/home/workspace/torch-triton/triton-cpu/python/triton/_C', '-L/usr/local/lib', '-I/usr/local/include', '-I/tmp/tmp59kzdlsc', '-I/usr/local/include/python3.10', '-Wl,-rpath', PosixPath('/home/workspace/torch-triton/triton-cpu/python/triton/_C'), '-Wl,-rpath', '/usr/local/lib', '-g']' returned non-zero exit status 1.

Environment details

use the newest master && cpu backend test

[krishagarwal]

I found the same issue. Looking further up in the error log, I saw the following message from the assembler:

/tmp/tmpfne6g4g3/kernel.s: Assembler messages: /tmp/tmpfne6g4g3/kernel.s:4363: Error: junk at end of line, first unrecognized character is `"'

I manually printed the assembly source from python (TRITON_KERNEL_DUMP=1 TRITON_DUMP_DIR=<some directory> wasn't working) and saw that the line causing the error is:

.file	1 "<path>/triton-cpu/python/tutorials" "01-vector-add.py"

I guess the assembler needs this path to be one string rather than split into two, meaning it should be

.file	1 "<path>/triton-cpu/python/tutorials/01-vector-add.py"

For now, as a really dirty fix, I inserted the following at line 93 in python/triton/runtime/build.py:

with open(src, "r") as f:
    src_txt = f.read()
src_txt = src_txt.replace('" "', "/")
with open(src, "w") as f:
    f.write(src_txt)

Basically, this will manually combine the path into one string in the assembly source before compiling. It seems to be working fine for me so far, although I would obviously hope a more proper fix is implemented.

[ienkovich]

I've seen such a problem previously. It was caused by as not recognizing .file directive and we've fixed it by adding -g compilation option. Initially, we used -gdwarf-5, but later moved to -g (IIRC -gdwarf-5 caused failure on older toolchains). So I suspect the issue might be related to the binary toolchain version on your system. What is our toolchain version? Can you successfully compile the original asm file with your gcc/as providing -gdwarf-5 options (asm compilation options are added at python/triton/runtime/build.py:92)?

[krishagarwal]

gcc version 9.4.0, as version 2.34. The issue persists even after using -gdwarf-5.

[ienkovich]

In our flows we use GCC 11, Binutils 2.38 or newer, so upgrading should resolve the issue.

A possible workaround is to use TRITON_DISABLE_LINE_INFO=1. This should remove all .file directives you have a problem with.

[GodHforever]

Met same problem，workaround TRITON_DISABLE_LINE_INFO=1 is useful.

Under gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

[Artlesbol]

I meet same problem under gcc-11. @ienkovich
Are there any other possible solutions?

$ /usr/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-2ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-PfdVzN/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-PfdVzN/gcc-11-11.4.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-2ubuntu1~20.04) 

when I run

$python /home/llb/bp_test/triton.py

I found the same issue.

/tmp/tmpa461syu_/kernel.s: Assembler messages:
/tmp/tmpa461syu_/kernel.s:1163: Error: junk at end of line, first unrecognized character is `"'
Traceback (most recent call last):
  File "/home/llb/bp_test/triton_cpu.py", line 59, in <module>
    output_triton = add(x, y)
  File "/home/llb/bp_test/triton_cpu.py", line 49, in add
    add_kernel[grid](x, y, output, n_elements, BLOCK_SIZE=1024)
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/runtime/jit.py", line 330, in <lambda>
    return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/runtime/jit.py", line 630, in run
    kernel = self.compile(
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/compiler/compiler.py", line 286, in compile
    next_module = compile_ir(module, metadata)
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/backends/cpu/compiler.py", line 311, in <lambda>
    stages["so"] = lambda src, metadata: self.make_so(src, metadata, options)
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/backends/cpu/compiler.py", line 301, in make_so
    so = _build("kernel", asm_path, tmpdir, lib_dirs, cpu_driver.include_dirs, libs)
  File "/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/runtime/build.py", line 96, in _build
    ret = subprocess.check_call(cc_cmd)
  File "/home/llb/anaconda3/envs/triton-benchmark-cases/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcc', '/tmp/tmpa461syu_/kernel.s', '-O3', '-shared', '-fPIC', '-Wno-psabi', '-o', '/tmp/tmpa461syu_/kernel.cpython-310-x86_64-linux-gnu.so', '-lm', '-lTritonCPURuntime', '-lsleef', '-lgcc', '-L/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/_C', '-L/usr/local/lib', '-I/usr/local/include', '-I/tmp/tmpa461syu_', '-I/home/llb/anaconda3/envs/triton-benchmark-cases/include/python3.10', '-Wl,-rpath', PosixPath('/home/llb/PLCT-interns/triton-benchmark/benchmarks/triton-cpu/python/triton/_C'), '-Wl,-rpath', '/usr/local/lib', '-g']' returned non-zero exit status 1.

The workaround TRITON_DISABLE_LINE_INFO=1 works for me

[sheenisme]

I also encountered this problem, have you solved it? TRITON_DISABLE_LINE_INFO=1 is useful for me.

$ gcc --version
gcc (GCC) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.