adding amdgpu ext skeleton.

[cmhamel]

Accidentally deleted the old PR.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 23 lines in your changes missing coverage. Please review.

Project coverage is 60.95%. Comparing base (f235837) to head (57991d1).
Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
src/DofManagers.jl	0.00%	15 Missing ⚠️
ext/FiniteElementContainersAMDGPUExt.jl	0.00%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #88      +/-   ##
==========================================
- Coverage   61.81%   60.95%   -0.87%     
==========================================
  Files          35       36       +1     
  Lines        1616     1639      +23     
==========================================
  Hits          999      999              
- Misses        617      640      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lxmota]

The syntax looks ok to me and it compiled on my RX 7600 GPU. Now I need to figure out how to test it.

[cmhamel]

The syntax looks ok to me and it compiled on my RX 7600 GPU. Now I need to figure out how to test it.

I added an example under test/poisson/TestPoissonAMDGPU.jl. The assembler seems to be working, but I'm getting some HIP errors in the cg solver.

This was tested on a RX 7900 I picked up yesterday. Maybe I still have some driver kinks to work out.

1-element ExceptionStack:
LoadError: HIPError(code hipErrorInvalidValue, one or more arguments outside of acceptable range)
Stacktrace:
  [1] check(err::AMDGPU.HIP.hipError_t)
    @ AMDGPU.HIP ~/.julia/packages/AMDGPU/EcH58/src/hip/error.jl:145
  [2] macro expansion
    @ ~/.julia/packages/AMDGPU/EcH58/src/utils.jl:123 [inlined]
  [3] hipModuleLaunchKernel
    @ ~/.julia/packages/AMDGPU/EcH58/src/hip/libhip.jl:287 [inlined]
  [4] (::AMDGPU.Runtime.var"#24#25"{Int64, HIPStream, AMDGPU.HIP.HIPFunction, AMDGPU.Runtime.ROCDim3, AMDGPU.Runtime.ROCDim3})(kernel_params::Vector{Ptr{Nothing}})
    @ AMDGPU.Runtime ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:124
  [5] macro expansion
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:110 [inlined]
  [6] macro expansion
    @ ./none:0 [inlined]
  [7] pack_arguments(::AMDGPU.Runtime.var"#24#25"{Int64, HIPStream, AMDGPU.HIP.HIPFunction, AMDGPU.Runtime.ROCDim3, AMDGPU.Runtime.ROCDim3}, ::AMDGPU.KernelState, ::AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}, ::AMDGPU.Device.ROCDeviceVector{Int32, 1}, ::AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1})
    @ AMDGPU.Runtime ./none:0
  [8] #launch#23
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:123 [inlined]
  [9] launch
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:116 [inlined]
 [10] #18
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:85 [inlined]
 [11] macro expansion
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:78 [inlined]
 [12] macro expansion
    @ ./none:0 [inlined]
 [13] convert_arguments
    @ ./none:0 [inlined]
 [14] #roccall#17
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:86 [inlined]
 [15] roccall
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:84 [inlined]
 [16] macro expansion
    @ ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:50 [inlined]
 [17] macro expansion
    @ ./none:0 [inlined]
 [18] call(::AMDGPU.Runtime.HIPKernel{typeof(AMDGPU.rocSPARSE.sparse_to_sparse_broadcast_kernel), Tuple{typeof(-), AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}, AMDGPU.Device.ROCDeviceVector{Int32, 1}, AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}}}, ::typeof(-), ::AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}, ::AMDGPU.Device.ROCDeviceVector{Int32, 1}, ::AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}; stream::HIPStream, call_kwargs::@Kwargs{gridsize::Int64, groupsize::Int64})
    @ AMDGPU.Runtime ./none:0
 [19] call
    @ ./none:0 [inlined]
 [20] (::AMDGPU.Runtime.HIPKernel{typeof(AMDGPU.rocSPARSE.sparse_to_sparse_broadcast_kernel), Tuple{typeof(-), AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}, AMDGPU.Device.ROCDeviceVector{Int32, 1}, AMDGPU.rocSPARSE.ROCSparseDeviceMatrixCSC{Float64, Int32, 1}}})(::Function, ::AMDGPU.rocSPARSE.ROCSparseMatrixCSC{Float64, Int32}, ::ROCArray{Int32, 1, AMDGPU.Runtime.Mem.HIPBuffer}, ::AMDGPU.rocSPARSE.ROCSparseMatrixCSC{Float64, Int32}; stream::HIPStream, call_kwargs::@Kwargs{gridsize::Int64, groupsize::Int64})
    @ AMDGPU.Runtime ~/.julia/packages/AMDGPU/EcH58/src/runtime/hip-execution.jl:59
 [21] macro expansion
    @ ~/.julia/packages/AMDGPU/EcH58/src/highlevel.jl:156 [inlined]
 [22] copy(bc::Base.Broadcast.Broadcasted{AMDGPU.rocSPARSE.ROCSparseMatStyle, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, typeof(-), Tuple{AMDGPU.rocSPARSE.ROCSparseMatrixCSC{Float64, Int32}}})
    @ AMDGPU.rocSPARSE ~/.julia/packages/AMDGPU/EcH58/src/sparse/broadcast.jl:583
 [23] materialize
    @ ./broadcast.jl:903 [inlined]
 [24] broadcast_preserving_zero_d
    @ ./broadcast.jl:892 [inlined]
 [25] -
    @ ./abstractarraymath.jl:218 [inlined]
 [26] poisson_amdgpu()
    @ Main ~/dev/cthonios/FiniteElementContainers.jl/test/poisson/TestPoissonAMDGPU.jl:80
 [27] macro expansion
    @ ./timing.jl:279 [inlined]
 [28] top-level scope
    @ ~/dev/cthonios/FiniteElementContainers.jl/test/poisson/TestPoissonAMDGPU.jl:269
 [29] include(fname::String)
    @ Base.MainInclude ./client.jl:494
 [30] top-level scope
    @ REPL[19]:1
in expression starting at /home/craig/dev/cthonios/FiniteElementContainers.jl/test/poisson/TestPoissonAMDGPU.jl:102

[cmhamel]

@lxmota the AMDGPU v1.3 release seems to have fixed the issues I had on my machine. Looks like we're up and running for both CUDA and AMD. Metal isn't even worth talking about until Apple decides to support double precision.