Skip to content

Multiple Helix work items fail on maccatalyst/tvos CoreCLR Release #126460

Description

@kotlarmilos

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1362636
Build error leg or test failing: Microsoft.Extensions.DependencyInjection.ExternalContainers.Tests.WorkItemExecution, Microsoft.Extensions.Hosting.Unit.Tests.WorkItemExecution, System.Diagnostics.TraceSource.Config.Tests.WorkItemExecution, System.Reflection.Emit.ILGeneration.Tests.WorkItemExecution, System.Reflection.Emit.Tests.WorkItemExecution, System.Reflection.Metadata.Tests.WorkItemExecution, System.Security.Cryptography.Tests.WorkItemExecution
Pull request: #125441

Affected CI jobs: build_MacCatalyst_x64_Release_AllSubsets_CoreCLR, build_maccatalyst_arm64_Release_AllSubsets_CoreCLR, build_tvos_arm64_Release_AllSubsets_CoreCLR, build_tvos_arm64_Release_AllSubsets_CoreCLR_RuntimeTests, build_tvos_arm64_Release_AllSubsets_Mono

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item (Microsoft\\.Extensions\\.DependencyInjection\\.ExternalContainers\\.Tests|Microsoft\\.Extensions\\.Hosting\\.Unit\\.Tests|System\\.Diagnostics\\.TraceSource\\.Config\\.Tests|System\\.Reflection\\.Emit\\.ILGeneration\\.Tests|System\\.Reflection\\.Emit\\.Tests|System\\.Reflection\\.Metadata\\.Tests|System\\.Security\\.Cryptography\\.Tests|Microsoft\\.Bcl\\.Cryptography\\.Tests|Microsoft\\.Bcl\\.Memory\\.Tests) in job [0-9a-f-]+ has failed\\. \\[/Users/runner/[^\\]]*sendtohelixhelp\\.proj\\]",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Summary

Multiple Helix work items fail on maccatalyst CoreCLR Release (both x64 and arm64) and tvos-arm64 CoreCLR/Mono Release, including the tvos-arm64 device queue OSX.15.Amd64.AppleTV.Open. Work items crash without producing individual test results (Build Analysis reports DeadLetter or BadExit).

Affected work items:

  • Microsoft.Extensions.DependencyInjection.ExternalContainers.Tests
  • Microsoft.Extensions.Hosting.Unit.Tests
  • System.Diagnostics.TraceSource.Config.Tests
  • System.Reflection.Emit.ILGeneration.Tests
  • System.Reflection.Emit.Tests
  • System.Reflection.Metadata.Tests
  • System.Security.Cryptography.Tests
  • Microsoft.Bcl.Cryptography.Tests (tvos-arm64 CoreCLR + Mono, DeadLetter, exit -3 / 21)
  • Microsoft.Bcl.Memory.Tests (tvos-arm64 CoreCLR, BadExit, exit 21)

Note: This issue uses timeline ErrorPattern matching because Helix console logs for this build expired. Once a fresh build reproduces this failure, update with specific error text from the Helix console log and set ExcludeConsoleLog to false.

Update (2026-06-17, build 1468533): added baseservices-exceptions, Exceptions, Microsoft.Bcl.Cryptography.Tests, and Microsoft.Bcl.Memory.Tests to the ErrorPattern; these recur on the tvos-arm64 OSX.15.Amd64.AppleTV.Open device queue (same crash family) but were not previously attributed by Build Analysis.

Update (2026-07-07): narrowed the ErrorPattern to Apple mobile CoreCLR library legs only. The previous pattern matched Work item <name> in job .* has failed on any leg, so it attributed unrelated desktop failures. The generic CoreCLR runtime-test suite names Exceptions and baseservices-exceptions run on every platform through src/tests/Common/helixpublishwitharcade.proj and matched desktop Linux and Windows runtime-test legs (for example build 1494680 on Windows and build 1495630 on Linux). The remaining library work items also run on all platforms. The pattern now requires the macOS runner path /Users/runner/... and the library helix project src/libraries/sendtohelixhelp.proj, which excludes Linux (/__w/) and Windows (D:\a\) hosts and the runtime-test project. Exceptions and baseservices-exceptions are dropped because they only appear on helixpublishwitharcade.proj, which cannot be distinguished from osx-x64 desktop runtime tests on a single log line. If tvos-arm64 runtime-test crashes need attribution, match a mobile-specific crash signature from the Helix console log instead.

Note

This issue was filed with the assistance of GitHub Copilot.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1496248
Error message validated: [Work item (Microsoft\.Extensions\.DependencyInjection\.ExternalContainers\.Tests|Microsoft\.Extensions\.Hosting\.Unit\.Tests|System\.Diagnostics\.TraceSource\.Config\.Tests|System\.Reflection\.Emit\.ILGeneration\.Tests|System\.Reflection\.Emit\.Tests|System\.Reflection\.Metadata\.Tests|System\.Security\.Cryptography\.Tests|Microsoft\.Bcl\.Cryptography\.Tests|Microsoft\.Bcl\.Memory\.Tests) in job [0-9a-f-]+ has failed\. \[/Users/runner/[^\]]*sendtohelixhelp\.proj\]]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/7/2026 12:34:07 PM UTC

Report

Build Repository Step Name Console log Pull Request
1497883 dotnet/runtime Send to Helix Log #130168
1496784 dotnet/runtime Send to Helix Log #130292
1496783 dotnet/runtime Send to Helix Log #130168
1496347 dotnet/runtime Send to Helix Log
1496248 dotnet/runtime Send to Helix Log #124721
1496266 dotnet/runtime Send to Helix Log
1495952 dotnet/runtime Send to Helix Log
1495630 dotnet/runtime Send tests to Helix (Unix) Log #130142
1493534 dotnet/runtime Send tests to Helix (Unix) Log #130141
1495180 dotnet/runtime Send to Helix Log
1495192 dotnet/runtime Send to Helix Log
1495016 dotnet/runtime Send to Helix Log
1494859 dotnet/runtime Send to Helix Log #130232
1494829 dotnet/runtime Send to Helix Log
1494680 dotnet/runtime Send tests to Helix (Windows) Log #129992
1494534 dotnet/runtime Send to Helix Log
1494467 dotnet/runtime Send to Helix Log #130221
1494308 dotnet/runtime Send to Helix Log
1494206 dotnet/runtime Send to Helix Log #130156
1494157 dotnet/runtime Send tests to Helix (Unix) Log #130203
1494160 dotnet/runtime Send to Helix Log
1494025 dotnet/runtime Send tests to Helix (Windows) Log
1493986 dotnet/runtime Send to Helix Log
1493971 dotnet/runtime Send tests to Helix (Unix) Log #129992
1493844 dotnet/runtime Send to Helix Log #130156
1493751 dotnet/runtime Send to Helix Log #128799
1493574 dotnet/runtime Send tests to Helix (Unix) Log #130172
1493403 dotnet/runtime Send to Helix Log #130156
1493092 dotnet/runtime Send to Helix Log #129845
1493324 dotnet/runtime Send tests to Helix (Unix) Log #130172
1493145 dotnet/runtime Send to Helix Log #130170
1492735 dotnet/runtime Send to Helix Log #130061
1492827 dotnet/runtime Send to Helix Log
1492684 dotnet/runtime Send to Helix Log
1492589 dotnet/runtime Send to Helix Log
1492674 dotnet/runtime Send to Helix Log #130156
1492240 dotnet/runtime Send to Helix Log
1492316 dotnet/runtime Send tests to Helix (Unix) Log #130141
1491753 dotnet/runtime Send to Helix Log #130116
1491060 dotnet/runtime Send to Helix Log #129609
1491027 dotnet/runtime Send to Helix Log
1490642 dotnet/runtime Send to Helix Log #130099
1490126 dotnet/runtime Send to Helix Log #127617
1489740 dotnet/runtime Send to Helix Log #130077
3012445 dotnet-runtime Send to Helix Log #62402
1489396 dotnet/runtime Send to Helix Log
1489109 dotnet/runtime Send to Helix Log
1488812 dotnet/runtime Send to Helix Log #130058
1488421 dotnet/runtime Send to Helix Log #130049
1488318 dotnet/runtime Send tests to Helix (Unix) Log
1488281 dotnet/runtime Send tests to Helix (Windows) Log #129474
1487856 dotnet/runtime Send to Helix Log #125437
1487904 dotnet/runtime Send tests to Helix (Unix) Log #130034
1487571 dotnet/runtime Send to Helix Log
1487037 dotnet/runtime Send to Helix Log
1487022 dotnet/runtime Send to Helix Log #130011
1486838 dotnet/runtime Send tests to Helix (Windows) Log #129474
1486312 dotnet/runtime Send to Helix Log
1486105 dotnet/runtime Send to Helix Log #129979
1485789 dotnet/runtime Send to Helix Log
3010659 dotnet-runtime Send to Helix Log #62402
1485187 dotnet/runtime Send to Helix Log
1485378 dotnet/runtime Send to Helix Log #129582
1485343 dotnet/runtime Send tests to Helix (Unix) Log #129424
1485131 dotnet/runtime Send to Helix Log
1484633 dotnet/runtime Send to Helix Log
1484622 dotnet/runtime Send tests to Helix (Unix) Log #129941
1484402 dotnet/runtime Send to Helix Log
1484157 dotnet/runtime Send to Helix Log #129923
1484071 dotnet/runtime Send to Helix Log
1484052 dotnet/runtime Send to Helix Log #127980
1483966 dotnet/runtime Send to Helix Log
1483955 dotnet/runtime Send to Helix Log #129918
1483740 dotnet/runtime Send to Helix Log
1483366 dotnet/runtime Send to Helix Log #129255
1483622 dotnet/runtime Send tests to Helix (Windows) Log #129877
1483621 dotnet/runtime Send tests to Helix (Windows) Log #129877
1483605 dotnet/runtime Send to Helix Log #129877
1482396 dotnet/runtime Send to Helix Log
1482372 dotnet/runtime Send to Helix Log
1482333 dotnet/runtime Send tests to Helix (Windows) Log #129877
1482287 dotnet/runtime Send tests to Helix (Windows) Log #129877
1481946 dotnet/runtime Send to Helix Log
1481374 dotnet/runtime Send tests to Helix (Unix) Log #129851
1481260 dotnet/runtime Send to Helix Log #129688
3007965 dotnet-runtime Send to Helix Log #62402
1480969 dotnet/runtime Send tests to Helix (Unix) Log #129396
3007933 dotnet-runtime Send to Helix Log #62402
1480831 dotnet/runtime Send to Helix Log #129119
1480565 dotnet/runtime Send to Helix Log
1480312 dotnet/runtime Send to Helix Log
1480576 dotnet/runtime Send to Helix Log
1480439 dotnet/runtime Send to Helix Log
1480194 dotnet/runtime Send to Helix Log #129702
1479813 dotnet/runtime Send to Helix Log
1480185 dotnet/runtime Send tests to Helix (Unix) Log #129396
1479326 dotnet/runtime Send tests to Helix (Unix) Log #129396
1479030 dotnet/runtime Send to Helix Log #129713
1478989 dotnet/runtime Send tests to Helix (Unix) Log #129396
1478795 dotnet/runtime Send to Helix Log
Displaying 100 of 250 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
6 47 250

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions