Default to SharedStorage

[christiangnrd]

No description provided.

[github-actions]

Metal Benchmarks

Details

Benchmark suite	Current: fb928ff	Previous: b70608b	Ratio
array/accumulate/Float32/1d	406833 ns	420625 ns	0.97
array/accumulate/Float32/dims=1	379208 ns	394584 ns	0.96
array/accumulate/Float32/dims=1L	8796958 ns	8876375 ns	0.99
array/accumulate/Float32/dims=2	446791 ns	456459 ns	0.98
array/accumulate/Float32/dims=2L	2622292 ns	2084125 ns	1.26
array/accumulate/Int64/1d	852292 ns	875500 ns	0.97
array/accumulate/Int64/dims=1	941167 ns	739750 ns	1.27
array/accumulate/Int64/dims=1L	9526583 ns	9664459 ns	0.99
array/accumulate/Int64/dims=2	1257000 ns	1073583 ns	1.17
array/accumulate/Int64/dims=2L	6509375 ns	6758792 ns	0.96
array/broadcast	209416 ns	194208 ns	1.08
array/construct	2334 ns	2416 ns	0.97
array/permutedims/2d	392833 ns	461958 ns	0.85
array/permutedims/3d	1012333 ns	864167 ns	1.17
array/permutedims/4d	1153500 ns	1033917 ns	1.12
array/private/copy	229167 ns	238958 ns	0.96
array/private/copyto!/cpu_to_gpu	194125 ns	200000 ns	0.97
array/private/copyto!/gpu_to_cpu	196375 ns	204833 ns	0.96
array/private/copyto!/gpu_to_gpu	194292 ns	156167 ns	1.24
array/private/iteration/findall/bool	1060417 ns	1132875 ns	0.94
array/private/iteration/findall/int	1226458 ns	1261125 ns	0.97
array/private/iteration/findfirst/bool	1122083 ns	1093000 ns	1.03
array/private/iteration/findfirst/int	857333 ns	1121292 ns	0.76
array/private/iteration/findmin/1d	1231666 ns	1278916 ns	0.96
array/private/iteration/findmin/2d	1075084 ns	779792 ns	1.38
array/private/iteration/logical	1557750 ns	1854625 ns	0.84
array/private/iteration/scalar	1153125 ns	1200666 ns	0.96
array/random/rand/Float32	433500 ns	436250 ns	0.99
array/random/rand/Int64	518333 ns	543583 ns	0.95
array/random/rand!/Float32	314667 ns	378750 ns	0.83
array/random/rand!/Int64	400208 ns	405167 ns	0.99
array/random/randn/Float32	392417 ns	400708 ns	0.98
array/random/randn!/Float32	352375 ns	299666 ns	1.18
array/reductions/mapreduce/Float32/1d	258541 ns	392875 ns	0.66
array/reductions/mapreduce/Float32/dims=1	334584 ns	323750 ns	1.03
array/reductions/mapreduce/Float32/dims=1L	627500 ns	631500 ns	0.99
array/reductions/mapreduce/Float32/dims=2	278291 ns	330334 ns	0.84
array/reductions/mapreduce/Float32/dims=2L	992083 ns	768958 ns	1.29
array/reductions/mapreduce/Int64/1d	450167 ns	586833 ns	0.77
array/reductions/mapreduce/Int64/dims=1	624875 ns	588375 ns	1.06
array/reductions/mapreduce/Int64/dims=1L	1032833 ns	1044583 ns	0.99
array/reductions/mapreduce/Int64/dims=2	518334 ns	648250 ns	0.80
array/reductions/mapreduce/Int64/dims=2L	2183458 ns	2146084 ns	1.02
array/reductions/reduce/Float32/1d	257041 ns	387416 ns	0.66
array/reductions/reduce/Float32/dims=1	339333 ns	329542 ns	1.03
array/reductions/reduce/Float32/dims=1L	631958 ns	631583 ns	1.00
array/reductions/reduce/Float32/dims=2	234875 ns	198208 ns	1.18
array/reductions/reduce/Float32/dims=2L	444750 ns	441417 ns	1.01
array/reductions/reduce/Int64/1d	438084 ns	584917 ns	0.75
array/reductions/reduce/Int64/dims=1	629042 ns	605375 ns	1.04
array/reductions/reduce/Int64/dims=1L	1036250 ns	1037667 ns	1.00
array/reductions/reduce/Int64/dims=2	240917 ns	196875 ns	1.22
array/reductions/reduce/Int64/dims=2L	651625 ns	642500 ns	1.01
array/shared/copy	135583 ns	142125 ns	0.95
array/shared/copyto!/cpu_to_gpu	38167 ns	38250 ns	1.00
array/shared/copyto!/gpu_to_cpu	38959 ns	37916 ns	1.03
array/shared/copyto!/gpu_to_gpu	39125 ns	38292 ns	1.02
array/shared/iteration/findall/bool	929666 ns	1133500 ns	0.82
array/shared/iteration/findall/int	1116209 ns	1275041 ns	0.88
array/shared/iteration/findfirst/bool	961333 ns	785417 ns	1.22
array/shared/iteration/findfirst/int	657542 ns	800416 ns	0.82
array/shared/iteration/findmin/1d	1096916 ns	991542 ns	1.11
array/shared/iteration/findmin/2d	1077292 ns	760875 ns	1.42
array/shared/iteration/logical	1565334 ns	1714417 ns	0.91
array/shared/iteration/scalar	4059.4285714285716 ns	4136.857142857143 ns	0.98
array/sorting/1d	1620708 ns	1845875 ns	0.88
array/sorting/2d	8104459 ns	8334000 ns	0.97
integration/byval/reference	1108667 ns	1108208 ns	1.00
integration/byval/slices=1	1104291 ns	1103917 ns	1.00
integration/byval/slices=2	2022666 ns	2033125 ns	0.99
integration/byval/slices=3	6589292 ns	14292709 ns	0.46
integration/metaldevrt	310583 ns	315209 ns	0.99
kernel/indexing	184208 ns	175208 ns	1.05
kernel/indexing_checked	374666 ns	305250 ns	1.23
kernel/launch	1875 ns	1912.5 ns	0.98
kernel/rand	297875 ns	378292 ns	0.79
latency/import	2028049375 ns	2060685917 ns	0.98
latency/precompile	38628376250 ns	39136197959 ns	0.99
latency/ttfp	2346676458 ns	2388951334 ns	0.98
metal/synchronization/context	589.35 ns	590.2375690607735 ns	1.00
metal/synchronization/stream	339.2857142857143 ns	341.9357798165138 ns	0.99

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

Codecov Report

❌ Patch coverage is 42.85714% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.08%. Comparing base (b8d46ce) to head (4492e5f).

Files with missing lines	Patch %	Lines
src/Metal.jl	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #822      +/-   ##
==========================================
- Coverage   86.62%   86.08%   -0.54%     
==========================================
  Files          76       76              
  Lines        5144     5147       +3     
==========================================
- Hits         4456     4431      -25     
- Misses        688      716      +28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[christiangnrd]

Could the github actions failure be related to the compilation failures?

[maleadt]

I'm not sure how. There's a known issue with ObjC errors leaking outside of their retain/release scope and crashing during error reporting, but that's read-only and shouldn't cause a crash in LLVM.

[maleadt]

I guess the remaining question is semantics. Do we want to allow scalar iteration on shard memory so that it can be used with CPU code? It'll never be as fast as Array, but if we port the dirty memory flag tracking from CUDA.jl we can get it down to a couple of ns (as opposed to ~1ns for an Array getindex or so). If we want to add back scalar iteration checking we add another couple of ns for the TLS check on every access.

[christiangnrd]

i.e. unsafe_wrap(Array, ... wouldn't be needed?

[maleadt]

Right, that's the current design of CUDA.jl:

Precompiling CUDA finished.
  12 dependencies successfully precompiled in 43 seconds. 90 already precompiled.

julia> CUDA.allowscalar(false)

julia> a = cu([1])
1-element CuArray{Int64, 1, CUDACore.DeviceMemory}:
 1

julia> a[]
ERROR: Scalar indexing is disallowed.

julia> b = cu([1]; unified=true)
1-element CuArray{Int64, 1, CUDACore.UnifiedMemory}:
 1

julia> b[]
1

I'm not entirely convinced this is the best option though. It makes sense, but people often use allowscalar for detecting GPU code. We could tell them they have to use private memory for that, but it still makes allowscalar(false) a lie.

[christiangnrd]

This is also how shared storage currently works

[christiangnrd]

But without the dirty flag so there might be some latent race conditions with shared MtlArrays

[maleadt]

This is also how shared storage currently works

Right, but it never was the default. It would break users doing allowscalar(false) to detect GPU functionality execution on the CPU.

And we definitely need the dirty flag to improve performance here. But that can be follow-up work.

[christiangnrd]

What of we add a @warn when they call allowscalar the first time if default storage mode is shared?

[maleadt]

I guess that could work, but the interface is not extensible like that right now.

EDIT: I guess we could implement Metal.allowscalar, have it warn, and then call GPUArrays.allowscalar.

[christiangnrd]

New relevant case: unsupported sorts (by != identity) will silently run on the CPU under SharedStorage.