[CoreCLR] dlopen()-based assembly store loading

Documentation/project-docs/ApkSharedLibraries.md

@@ -72,6 +72,12 @@ those directories into actual ELF shared libraries.  The way it is done is descr
 
 ## Data payload stub library
 
+> **Note:** there are currently two payload-library layouts. The **MonoVM** layout described in this section
+> (a stub with a non-loadable `payload` section, located by scanning the APK ZIP and `mmap`ed by hand) and the
+> **CoreCLR** layout (a real shared library loaded via `dlopen()`+`dlsym()`), described in
+> [CoreCLR: dlopen-based payload library](#coreclr-dlopen-based-payload-library). Everything below this note
+> describes the MonoVM layout unless stated otherwise.
+
 ELF binaries consist of a number of sections, which contain code, data (read-only and read-write), debug symbols etc.
 However, the ELF specification doesn't dictate names of any of those sections and, thus, developers are free to lay out
 ELF binaries any way they see fit, as long as the binary conforms to the ELF specification and the operating system
@@ -94,7 +100,35 @@ One downside of this approach is that if one were to run the `llvm-strip` or `st
 shared libray, the `payload` section (as it uses a "non-standard" name) would be considered by the strip utility
 to be unnecessary and summarily removed.
 
-### Layout of the payload library
+The layout described above (a non-loadable `payload` section that the runtime finds by parsing the ELF section
+headers itself) is used by the **MonoVM** runtime, which also locates the wrapper library by scanning the APK/AAB
+ZIP central directory and then `mmap`s it.
+
+### CoreCLR: dlopen-based payload library
+
+The **CoreCLR** runtime uses a different, simpler mechanism.  Its prebuilt native host (`libmonodroid.so`) is
+shared by every application and build configuration, so it cannot rely on a baked-in `DT_NEEDED` dependency on the
+assembly store (Debug/FastDev builds don't ship one).  Instead, the assembly store wrapper library is produced so
+that the store payload lives in a **loadable** ELF section (`SHF_ALLOC`, covered by a `PT_LOAD` segment) that is
+pointed at by an exported dynamic symbol named `_assembly_store`.  At runtime the host simply calls
+`dlopen("libassembly-store.so", …)` followed by `dlsym(handle, "_assembly_store")` and lets the dynamic linker
+locate and map the payload out of the APK — there is no ZIP scanning and no manual ELF section-header parsing.
+
+Because the section is allocatable and referenced by a dynamic symbol, this layout survives `strip`/`llvm-strip`,
+unlike the MonoVM `payload` section described above.
+
+This wrapper is produced by
+[`DlopenAssemblyStoreGenerator`](../../src/Xamarin.Android.Build.Tasks/Utilities/DlopenAssemblyStoreGenerator.cs)
+(rather than `DSOWrapperGenerator`), which assembles a tiny `.incbin` stub with `llvm-mc` and links it into a
+shared object with `ld` (no `llvm-objcopy` and no clang are involved).  The section is still named `payload`, so
+the extraction command (`llvm-objcopy --dump-section=payload=…`) shown below works for both layouts; the two
+layouts are compared in detail in [Layout of the CoreCLR (dlopen) payload library](#layout-of-the-coreclr-dlopen-payload-library).
+
+### Layout of the MonoVM (stub) payload library
+
+> **Note:** the sample output in this section is the MonoVM stub layout, produced by `DSOWrapperGenerator`.
+> Its `payload` section is **non-loadable** (no `A` flag, `Address` `0`) and there is no `_assembly_store`
+> dynamic symbol. For the CoreCLR layout see the [next section](#layout-of-the-coreclr-dlopen-payload-library).
 
 In order to examine content of our "payload" ELF shared library, one can run the `llvm-readelf` utility which is
 shipped with the Android NDK (and also part of native developer tools on macOS and Linux distributions which have
@@ -188,3 +222,69 @@ $ hexdump -c -n 4 payload.bin
 0000000   X   A   B   A
 0000004
 ```
+
+### Layout of the CoreCLR (dlopen) payload library
+
+The CoreCLR wrapper differs from the MonoVM stub in two ways that matter to the runtime:
+
+  1. The `payload` section is **allocatable** (`SHF_ALLOC`, shown as the `A` flag) and is assigned a
+     virtual address, so it is covered by a `PT_LOAD` program header and mapped into memory by the
+     dynamic linker as part of `dlopen()`.
+  2. An exported dynamic symbol, `_assembly_store`, points at the beginning of the payload, so the runtime
+     can retrieve the payload address with a single `dlsym()` call.
+
+The section header listing shows the `payload` section carrying the `A` flag and a non-zero address (compare
+with the MonoVM listing above, where `payload` has no flags and address `0`):
+
+```shell
+$ llvm-readelf --section-headers libassembly-store.so
+Section Headers:
+  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
+  ...
+  [ 5] .dynstr           STRTAB          0000000000000290 000290 000026 00   A  0   0  1
+  [ 6] payload           PROGBITS        0000000000004000 004000 001004 00   A  0   0 16384
+  [ 7] .text             PROGBITS        0000000000009004 005004 000000 00  AX  0   0  4
+  [ 8] .dynamic          DYNAMIC         000000000000d008 005008 000080 10  WA  5   0  8
+  ...
+```
+
+The program headers confirm that the `payload` section is part of a read-only `PT_LOAD` segment, i.e. the
+dynamic linker maps it for us:
+
+```shell
+$ llvm-readelf --program-headers libassembly-store.so
+Program Headers:
+  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
+  ...
+  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x005004 0x005004 R   0x4000
+  ...
+
+ Section to Segment mapping:
+  Segment Sections...
+   01     .note.gnu.build-id .dynsym .gnu.hash .hash .dynstr payload
+  ...
+```
+
+Finally, the dynamic symbol table exposes `_assembly_store`, whose value (`0x4000`) is the virtual address of
+the `payload` section — this is exactly what `dlsym(handle, "_assembly_store")` returns at runtime:
+
+```shell
+$ llvm-readelf --dyn-symbols libassembly-store.so
+Symbol table '.dynsym' contains 2 entries:
+   Num:    Value          Size Type    Bind   Vis       Ndx Name
+     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   UND
+     1: 0000000000004000     0 NOTYPE  GLOBAL DEFAULT     6 _assembly_store
+```
+
+(The offsets and sizes above come from a tiny sample payload; a real assembly store's `payload` section will
+be much larger, but the flags, `PT_LOAD` membership and the `_assembly_store` symbol are what identify a valid
+CoreCLR wrapper.)
+
+Extraction works the same as for the MonoVM layout, since the section is still called `payload`:
+
+```shell
+$ llvm-objcopy --dump-section=payload=payload.bin libassembly-store.so
+$ hexdump -c -n 4 payload.bin
+0000000   X   A   B   A
+0000004
+```

Documentation/project-docs/AssemblyStores.md

@@ -59,6 +59,20 @@ An assembly store, however, needs to be mapped only once and any
 further operations are merely pointer arithmetic, making the process
 not only faster but also reducing the algorithm complexity to O(1).
 
+The way the store is located and mapped depends on the runtime:
+
+  - **MonoVM** locates the store by scanning the APK/AAB ZIP central
+    directory, then `mmap(2)`s the wrapper shared library and walks its
+    ELF section headers by hand to find the `payload` section.
+  - **CoreCLR** relies on the dynamic linker instead: the store is
+    wrapped in a shared library whose payload lives in a *loadable* ELF
+    section pointed at by the exported `_assembly_store` dynamic symbol.
+    The runtime simply `dlopen()`s `libassembly-store.so` and resolves
+    the payload pointer with `dlsym("_assembly_store")`; there is no
+    ZIP scanning or manual section-header parsing. See
+    [ApkSharedLibraries.md](ApkSharedLibraries.md) for the two payload
+    layouts.
+
 # Store locations
 
 There exists only one Assembly Store per architecture. Each application will contain 

src/Xamarin.Android.Build.Tasks/Tasks/GenerateNativeApplicationConfigSources.cs

@@ -285,6 +285,7 @@ static bool ShouldSkipAssembly (ITaskItem assembly)
 					MarshalMethodsEnabled = EnableMarshalMethods,
 					ManagedMarshalMethodsLookupEnabled = EnableManagedMarshalMethodsLookup,
 					IgnoreSplitConfigs = ShouldIgnoreSplitConfigs (),
+					HaveAssemblyStore = UseAssemblyStore,
 				};
 			} else {
 				appConfigAsmGen = new ApplicationConfigNativeAssemblyGenerator (envBuilder.EnvironmentVariables, envBuilder.SystemProperties, Log) {

src/Xamarin.Android.Build.Tasks/Tasks/WrapAssembliesAsSharedLibraries.cs

@@ -33,6 +33,14 @@ public class WrapAssembliesAsSharedLibraries : AndroidTask
 
 	public bool UseAssemblyStore { get; set; }
 
+	/// <summary>
+	/// When true (always the case for CoreCLR), the assembly store is wrapped into a shared library
+	/// whose payload is exported via the <c>_assembly_store</c> dynamic symbol, so the runtime
+	/// can locate it with <c>dlopen</c>+<c>dlsym</c> instead of parsing the APK ZIP directory.
+	/// When false (MonoVM), the classic <see cref="DSOWrapperGenerator"/> layout is used instead.
+	/// </summary>
+	public bool UseDlopenAssemblyStore { get; set; }
+
 	[Required]
 	public ITaskItem [] ResolvedAssemblies { get; set; } = [];
 
@@ -69,7 +77,9 @@ void WrapAssemblyStores (DSOWrapperGenerator.Config dsoWrapperConfig, PackageFil
 
 			var arch = MonoAndroidHelper.AbiToTargetArch (abi);
 			var archive_path = MakeArchiveLibPath (abi, "lib" + Path.GetFileName (store_path));
-			var wrapped_source_path = DSOWrapperGenerator.WrapIt (Log, dsoWrapperConfig, arch, store_path, Path.GetFileName (archive_path));
+			var wrapped_source_path = UseDlopenAssemblyStore
+				? DlopenAssemblyStoreGenerator.WrapIt (Log, dsoWrapperConfig, arch, store_path, Path.GetFileName (archive_path))
+				: DSOWrapperGenerator.WrapIt (Log, dsoWrapperConfig, arch, store_path, Path.GetFileName (archive_path));
 
 			files.AddItem (wrapped_source_path, archive_path);
 		}

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateNativeApplicationConfigSourcesTests.cs

@@ -0,0 +1,45 @@
+#nullable enable
+using System.IO;
+
+using Microsoft.Build.Utilities;
+using NUnit.Framework;
+using Xamarin.Android.Tasks;
+using Xamarin.ProjectTools;
+
+namespace Xamarin.Android.Build.Tests.Tasks;
+
+[TestFixture]
+public class GenerateNativeApplicationConfigSourcesTests : BaseTest
+{
+	[TestCase (false)]
+	[TestCase (true)]
+	public void HaveAssemblyStoreIsEmittedForCoreCLR (bool haveAssemblyStore)
+	{
+		string outputRoot = Path.Combine (Root, "temp", $"{nameof (HaveAssemblyStoreIsEmittedForCoreCLR)}-{haveAssemblyStore}");
+		string monoAndroidPath = Path.Combine (TestEnvironment.MonoAndroidFrameworkDirectory, "Mono.Android.dll");
+		FileAssert.Exists (monoAndroidPath);
+
+		var task = new GenerateNativeApplicationConfigSources {
+			BuildEngine = new MockBuildEngine (TestContext.Out),
+			ResolvedAssemblies = [new TaskItem (monoAndroidPath)],
+			EnvironmentOutputDirectory = Path.Combine (outputRoot, "android"),
+			SupportedAbis = ["arm64-v8a"],
+			AndroidPackageName = "com.microsoft.android.assemblystoretest",
+			EnablePreloadAssembliesDefault = false,
+			TargetsCLR = true,
+			AndroidRuntime = "CoreCLR",
+			UseAssemblyStore = haveAssemblyStore,
+		};
+
+		Assert.IsTrue (task.Execute (), "GenerateNativeApplicationConfigSources should succeed.");
+
+		var environmentFiles = EnvironmentHelper.GatherEnvironmentFiles (
+			outputRoot,
+			"arm64-v8a",
+			required: true,
+			runtime: AndroidRuntime.CoreCLR
+		);
+		var config = (EnvironmentHelper.ApplicationConfig_CoreCLR)EnvironmentHelper.ReadApplicationConfig (environmentFiles, AndroidRuntime.CoreCLR);
+		Assert.AreEqual (haveAssemblyStore, config.have_assembly_store);
+	}
+}

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs

@@ -62,9 +62,10 @@ public sealed class ApplicationConfig_CoreCLR : IApplicationConfig
 			public uint   jni_remapping_replacement_method_index_entry_count;
 			public string android_package_name = String.Empty;
 			public bool   managed_marshal_methods_lookup_enabled;
+			public bool   have_assembly_store;
 		}
 
-		const uint ApplicationConfigFieldCount_CoreCLR = 20;
+		const uint ApplicationConfigFieldCount_CoreCLR = 21;
 
 		// This must be identical to the ApplicationConfig structure in src/native/mono/xamarin-app-stub/xamarin-app.hh
 		public sealed class ApplicationConfig_MonoVM : IApplicationConfig
@@ -407,6 +408,11 @@ static IApplicationConfig ReadApplicationConfig_CoreCLR (EnvironmentFile envFile
 						AssertFieldType (envFile.Path, parser.SourceFilePath, ".byte", field [0], item.LineNumber);
 						ret.managed_marshal_methods_lookup_enabled = ConvertFieldToBool ("managed_marshal_methods_lookup_enabled", envFile.Path, parser.SourceFilePath, item.LineNumber, field [1]);
 						break;
+
+					case 20: // have_assembly_store: bool / .byte
+						AssertFieldType (envFile.Path, parser.SourceFilePath, ".byte", field [0], item.LineNumber);
+						ret.have_assembly_store = ConvertFieldToBool ("have_assembly_store", envFile.Path, parser.SourceFilePath, item.LineNumber, field [1]);
+						break;
 				}
 				fieldCount++;
 			}
@@ -771,6 +777,7 @@ static void AssertApplicationConfigIsIdentical (ApplicationConfig_CoreCLR firstA
 			Assert.AreEqual (firstAppConfig.environment_variable_count, secondAppConfig.environment_variable_count, $"Field 'environment_variable_count' has different value in environment file '{secondEnvFile}' than in environment file '{firstEnvFile}'");
 			Assert.AreEqual (firstAppConfig.system_property_count, secondAppConfig.system_property_count, $"Field 'system_property_count' has different value in environment file '{secondEnvFile}' than in environment file '{firstEnvFile}'");
 			Assert.AreEqual (firstAppConfig.android_package_name, secondAppConfig.android_package_name, $"Field 'android_package_name' has different value in environment file '{secondEnvFile}' than in environment file '{firstEnvFile}'");
+			Assert.AreEqual (firstAppConfig.have_assembly_store, secondAppConfig.have_assembly_store, $"Field 'have_assembly_store' has different value in environment file '{secondEnvFile}' than in environment file '{firstEnvFile}'");
 		}
 
 		static void AssertApplicationConfigIsIdentical (ApplicationConfig_MonoVM firstAppConfig, string firstEnvFile, ApplicationConfig_MonoVM secondAppConfig, string secondEnvFile)

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/BuildReleaseArm64SimpleDotNet.CoreCLR.apkdesc

@@ -5,34 +5,34 @@
       "Size": 3036
     },
     "classes.dex": {
-      "Size": 402352
+      "Size": 402852
     },
     "lib/arm64-v8a/libassembly-store.so": {
-      "Size": 2717688
+      "Size": 2725488
     },
     "lib/arm64-v8a/libclrjit.so": {
-      "Size": 2835128
+      "Size": 2757816
     },
     "lib/arm64-v8a/libcoreclr.so": {
-      "Size": 4891056
+      "Size": 4837240
     },
     "lib/arm64-v8a/libmonodroid.so": {
-      "Size": 1324320
+      "Size": 1235528
     },
     "lib/arm64-v8a/libSystem.Globalization.Native.so": {
       "Size": 72112
     },
     "lib/arm64-v8a/libSystem.IO.Compression.Native.so": {
-      "Size": 1262888
+      "Size": 1258776
     },
     "lib/arm64-v8a/libSystem.Native.so": {
-      "Size": 101888
+      "Size": 99664
     },
     "lib/arm64-v8a/libSystem.Security.Cryptography.Native.Android.so": {
-      "Size": 168080
+      "Size": 163936
     },
     "lib/arm64-v8a/libxamarin-app.so": {
-      "Size": 20968
+      "Size": 21288
     },
     "res/drawable-hdpi-v4/icon.png": {
       "Size": 2178
@@ -59,5 +59,5 @@
       "Size": 1904
     }
   },
-  "PackageSize": 7271867
+  "PackageSize": 7235003
 }

src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigCLR.cs

@@ -51,4 +51,5 @@ sealed class ApplicationConfigCLR
 	public uint   jni_remapping_replacement_method_index_entry_count;
 	public string android_package_name = String.Empty;
 	public bool   managed_marshal_methods_lookup_enabled;
+	public bool   have_assembly_store;
 }