Enable and package the C# language extension on Linux

language-extensions/dotnet-core-CSharp/README.md

@@ -5,11 +5,11 @@ Language Extensions is a feature of SQL Server used for executing external code.
 
 For more information about SQL Server Language Extensions, refer to this [documentation](https://docs.microsoft.com/en-us/sql/language-extensions/language-extensions-overview?view=sql-server-ver15).
 
-The dotnet-core-CSharp-extension version in this repository is compatible with SQL Server 2019 CU3 onwards. It integrates .NET core in SQL Server and works with .NET 8.0 in **Windows only**.
+The dotnet-core-CSharp-extension version in this repository is compatible with SQL Server 2019 CU3 onwards. It integrates .NET Core in SQL Server and works with .NET 8.0 and up on Windows and Linux.
 
 Currently, the extension supports the following data types: SQL_C_SLONG, SQL_C_ULONG, SQL_C_SSHORT, SQL_C_USHORT, SQL_C_SBIGINT, SQL_C_UBIGINT, SQL_C_STINYINT, SQL_C_UTINYINT, SQL_C_BIT, SQL_C_FLOAT, SQL_C_DOUBLE, SQL_C_CHAR, SQL_C_WCHAR, and SQL_C_NUMERIC. It supports the following SQL data types: int, bigint, smallint, tinyint, real, float, bit, char(n), varchar(n), nchar(n), nvarchar(n), decimal(p,s), and numeric(p,s).
 
-To use this dotnet-core-CSharp-lang-extension.zip package, follow [this tutorial](./sample/regex/README.md). For any fixes or enhancements, you are welcome to modify, rebuild and use the binaries using the following instructions.
+To use this `dotnet-core-CSharp-lang-extension.zip` (Windows) or `dotnet-core-CSharp-lang-extension.tar.gz` (Linux) package, follow [this tutorial](./sample/regex/README.md). For any fixes or enhancements, you are welcome to modify, rebuild and use the binaries using the following instructions.
 
 ## Building
 
@@ -24,15 +24,24 @@ To use this dotnet-core-CSharp-lang-extension.zip package, follow [this tutorial
         - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\nativecsharpextension.dll \
         - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\hostfxr.dll \
         - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\Microsoft.SqlServer.CSharpExtension.dll \
-        - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\Microsoft.SqlServer.CSharpExtension.runtimeconfig.json\
+        - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\Microsoft.SqlServer.CSharpExtension.runtimeconfig.json \
         - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\Microsoft.SqlServer.CSharpExtension.deps.json
 
 5. Run [create-dotnet-core-CSharp-extension-zip.cmd](./build/windows/create-dotnet-core-CSharp-extension-zip.cmd) which will generate: \
-        - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\target\debug\dotnet-core-CSharp-lang-extension.zip
+        - PATH\TO\ENLISTMENT\build-output\dotnet-core-CSharp-extension\windows\release\packages\dotnet-core-CSharp-lang-extension.zip
         This zip can be used in CREATE EXTERNAL LANGUAGE, as detailed in the tutorial in the Usage section below.
 
 ### Linux
-Not Supported.
+
+1. Run [build-dotnet-core-CSharp-extension.ps1](./build/linux/build-dotnet-core-CSharp-extension.ps1) which will generate: \
+        - PATH/TO/ENLISTMENT/build-output/dotnet-core-CSharp-extension/linux/release/libnativecsharpextension.so \
+        - PATH/TO/ENLISTMENT/build-output/dotnet-core-CSharp-extension/linux/release/Microsoft.SqlServer.CSharpExtension.dll \
+        - PATH/TO/ENLISTMENT/build-output/dotnet-core-CSharp-extension/linux/release/Microsoft.SqlServer.CSharpExtension.runtimeconfig.json \
+        - PATH/TO/ENLISTMENT/build-output/dotnet-core-CSharp-extension/linux/release/Microsoft.SqlServer.CSharpExtension.deps.json
+
+2. Run [create-dotnet-core-CSharp-extension-zip.ps1](./build/linux/create-dotnet-core-CSharp-extension-zip.ps1) which will generate: \
+        - PATH/TO/ENLISTMENT/build-output/dotnet-core-CSharp-extension/linux/release/packages/dotnet-core-CSharp-lang-extension.tar.gz
+        This tarball can be used in CREATE EXTERNAL LANGUAGE, as detailed in the tutorial in the Usage section below.
 
 ## Testing (Optional)
 

language-extensions/dotnet-core-CSharp/build/linux/build-dotnet-core-CSharp-extension.sh

@@ -0,0 +1,178 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Set root and working directories
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENL_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
+DOTNET_EXTENSION_HOME="$ENL_ROOT/language-extensions/dotnet-core-CSharp"
+DOTNET_EXTENSION_WORKING_DIR="$ENL_ROOT/build-output/dotnet-core-CSharp-extension/linux"
+
+# Clean and create working directory
+rm -rf "$DOTNET_EXTENSION_WORKING_DIR"
+mkdir -p "$DOTNET_EXTENSION_WORKING_DIR"
+
+# Default to release if no arguments provided
+if [ $# -eq 0 ]; then
+    set -- "release"
+fi
+
+# Process each build configuration
+for BUILD_CONFIGURATION in "$@"; do
+    BUILD_CONFIGURATION="$(echo "$BUILD_CONFIGURATION" | tr '[:upper:]' '[:lower:]')"
+
+    # Default to release if not debug
+    if [ "$BUILD_CONFIGURATION" != "debug" ]; then
+        BUILD_CONFIGURATION="release"
+    fi
+
+    echo "[Info] Building dotnet-core-CSharp-extension libnativecsharpextension.so..."
+
+    # Set build output directory
+    BUILD_OUTPUT="$DOTNET_EXTENSION_WORKING_DIR/$BUILD_CONFIGURATION"
+    rm -rf "$BUILD_OUTPUT"
+    mkdir -p "$BUILD_OUTPUT"
+    pushd "$BUILD_OUTPUT" > /dev/null
+
+    # Set source and include paths
+    DOTNET_NATIVE_SRC="$DOTNET_EXTENSION_HOME/src/native"
+    DOTNET_NATIVE_INCLUDE="$DOTNET_EXTENSION_HOME/include"
+    EXTENSION_HOST_INCLUDE="$ENL_ROOT/extension-host/include"
+    DOTNET_NATIVE_LIB="$DOTNET_EXTENSION_HOME/lib"
+
+    # Determine C++ compiler.
+    # Prefer g++-11 (Ubuntu 22.04 toolchain) to match the SQL Server runtime
+    # container's glibc 2.35 / libstdc++ GLIBCXX_3.4.30 ABI. Newer compilers
+    # (Ubuntu 24.04 / gcc 13) generate references to GLIBC_2.38 symbols
+    # (e.g. __isoc23_strtoul) that don't exist in the runtime, causing
+    # dlopen() to fail silently and the extension to never load.
+    if [ -z "${CXX:-}" ]; then
+        if command -v g++-11 &>/dev/null; then
+            CXX="g++-11"
+        else
+            CXX="c++"
+        fi
+    fi
+    echo "[Info] Using C++ compiler: $CXX ($($CXX --version | head -1))"
+
+    # Build compiler arguments
+    CC_ARGS=(
+        -shared
+        -fPIC
+        -fshort-wchar
+        -std=c++17
+        -o libnativecsharpextension.so
+        "-I$DOTNET_NATIVE_INCLUDE"
+        "-I$EXTENSION_HOST_INCLUDE"
+        # Use g++-11's dynamic runtime. It stays within the RHEL 9/Ubuntu 22.04
+        # GLIBCXX ceiling without embedding a second C++ runtime and unwinder
+        # in the extension host process.
+    )
+
+    if [ "$BUILD_CONFIGURATION" = "debug" ]; then
+        CC_ARGS+=(-DDEBUG -g)
+    fi
+
+    # Add all .cpp source files
+    for src in "$DOTNET_NATIVE_SRC"/*.cpp; do
+        CC_ARGS+=("$src")
+    done
+
+    # Link with static nethost library (must run restore-packages.sh first)
+    if [ ! -f "$DOTNET_NATIVE_LIB/libnethost.a" ]; then
+        echo "Error: libnethost.a not found at $DOTNET_NATIVE_LIB/libnethost.a" >&2
+        echo "       Run restore-packages.sh first to install the .NET SDK and copy libnethost.a." >&2
+        exit 1
+    fi
+    CC_ARGS+=("$DOTNET_NATIVE_LIB/libnethost.a")
+    # Link with libdl for dlopen/dlsym
+    CC_ARGS+=(-ldl)
+
+    echo "[Info] Compiling with: $CXX ${CC_ARGS[*]}"
+    "$CXX" "${CC_ARGS[@]}"
+
+    popd > /dev/null
+
+    # Publish managed code as self-contained for linux-x64.
+    # This bundles the entire .NET runtime (hostfxr, coreclr, framework DLLs)
+    # into the output so the extension works without a system .NET installation.
+    echo "[Info] Publishing Microsoft.SqlServer.CSharpExtension (self-contained, linux-x64)..."
+    DOTNET_MANAGED_SRC="$DOTNET_EXTENSION_HOME/src/managed"
+    dotnet publish \
+        "$DOTNET_MANAGED_SRC/Microsoft.SqlServer.CSharpExtension.csproj" \
+        -c "$BUILD_CONFIGURATION" \
+        -r linux-x64 \
+        --self-contained \
+        -o "$BUILD_OUTPUT" \
+        --no-restore
+
+    # Post-process: transform the self-contained runtimeconfig.json for component hosting.
+    # hostfxr rejects self-contained component initialization (error 0x80008093) because
+    # runtimeconfig.json contains "includedFrameworks". We convert it to use "framework"
+    # (framework-dependent format) and create a shared/ symlink so hostfxr can resolve
+    # the bundled runtime via hostfxr_initialize_for_runtime_config.
+    RUNTIMECONFIG="$BUILD_OUTPUT/Microsoft.SqlServer.CSharpExtension.runtimeconfig.json"
+    if grep -q '"includedFrameworks"' "$RUNTIMECONFIG" 2>/dev/null; then
+        FRAMEWORK_VERSION=$(python3 -c "
+import json, sys
+with open(sys.argv[1]) as f:
+    cfg = json.load(f)
+ro = cfg.get('runtimeOptions', {})
+included = ro.get('includedFrameworks', [])
+if not included:
+    sys.exit(1)
+ro['framework'] = included[0]
+del ro['includedFrameworks']
+with open(sys.argv[1], 'w') as f:
+    json.dump(cfg, f, indent=2)
+print(included[0]['version'])
+" "$RUNTIMECONFIG")
+
+        if [ -n "$FRAMEWORK_VERSION" ]; then
+            # Create framework directory structure with file copies.
+            # hostfxr resolves frameworks at <dotnet_root>/shared/<framework>/<version>/
+            # which must contain the runtime DLLs and native libraries.
+            #
+            # We use file copies because SQL Server's internal tar extraction
+            # (used by CREATE EXTERNAL LANGUAGE) does not preserve symbolic links
+            # or hard links -- both are silently dropped during extraction.
+            # Copying the files ensures they are always present after extraction.
+            FRAMEWORK_DIR="$BUILD_OUTPUT/shared/Microsoft.NETCore.App/$FRAMEWORK_VERSION"
+            mkdir -p "$FRAMEWORK_DIR"
+            # Copy DLLs and SOs from the self-contained publish output to the framework dir.
+            for f in "$BUILD_OUTPUT"/*.dll "$BUILD_OUTPUT"/*.so; do
+                [ -e "$f" ] && cp "$f" "$FRAMEWORK_DIR/$(basename "$f")"
+            done
+            # hostfxr requires Microsoft.NETCore.App.deps.json to recognize a valid
+            # framework version directory. This file is NOT in the self-contained publish
+            # output -- it only exists in the .NET SDK's shared framework directory.
+            # Find and copy it from the SDK installation.
+            DOTNET_SDK_ROOT=$(dirname "$(dirname "$(command -v dotnet)")")
+            SDK_FX_DIR="$DOTNET_SDK_ROOT/shared/Microsoft.NETCore.App/$FRAMEWORK_VERSION"
+            if [ ! -d "$SDK_FX_DIR" ]; then
+                # Try the standard /usr/share/dotnet location
+                SDK_FX_DIR="/usr/share/dotnet/shared/Microsoft.NETCore.App/$FRAMEWORK_VERSION"
+            fi
+            if [ -f "$SDK_FX_DIR/Microsoft.NETCore.App.deps.json" ]; then
+                cp "$SDK_FX_DIR/Microsoft.NETCore.App.deps.json" "$FRAMEWORK_DIR/"
+                echo "  Copied Microsoft.NETCore.App.deps.json from SDK"
+            else
+                echo "WARNING: Microsoft.NETCore.App.deps.json not found in SDK at $SDK_FX_DIR"
+                # As a fallback, search for it under dotnet root
+                FOUND_DEPS=$(find /usr -name "Microsoft.NETCore.App.deps.json" -path "*/$FRAMEWORK_VERSION/*" 2>/dev/null | head -1)
+                if [ -n "$FOUND_DEPS" ]; then
+                    cp "$FOUND_DEPS" "$FRAMEWORK_DIR/"
+                    echo "  Found and copied deps.json from $FOUND_DEPS"
+                fi
+            fi
+            # Also copy .version if available
+            if [ -f "$SDK_FX_DIR/.version" ]; then
+                cp "$SDK_FX_DIR/.version" "$FRAMEWORK_DIR/"
+            fi
+            echo "Success: Configured for component hosting (framework $FRAMEWORK_VERSION, $(ls -a "$FRAMEWORK_DIR" | wc -l) files)"
+        fi
+    fi
+
+    echo "Success: Built dotnet-core-CSharp-extension for $BUILD_CONFIGURATION configuration."
+done
+
+exit 0

language-extensions/dotnet-core-CSharp/build/linux/create-dotnet-core-CSharp-extension-tar.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENL_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
+DOTNET_EXTENSION_WORKING_DIR="$ENL_ROOT/build-output/dotnet-core-CSharp-extension/linux"
+
+check_error() {
+    local error_level=$1
+    local error_message=$2
+    if [ "$error_level" -ne 0 ]; then
+        echo "Error: $error_message" >&2
+        exit "$error_level"
+    fi
+}
+
+# Default to release if no arguments provided
+if [ $# -eq 0 ]; then
+    set -- "release"
+fi
+
+# Process each build configuration
+for BUILD_CONFIGURATION in "$@"; do
+    BUILD_CONFIGURATION="$(echo "$BUILD_CONFIGURATION" | tr '[:upper:]' '[:lower:]')"
+
+    # Default to release if not debug
+    if [ "$BUILD_CONFIGURATION" != "debug" ]; then
+        BUILD_CONFIGURATION="release"
+    fi
+
+    BUILD_OUTPUT="$DOTNET_EXTENSION_WORKING_DIR/$BUILD_CONFIGURATION"
+    mkdir -p "$BUILD_OUTPUT/packages"
+
+    # Delete the ref folder so that the tarball can be loaded by the SPEES
+    rm -rf "$BUILD_OUTPUT/ref"
+
+    # Collect files to compress
+    FILES_TO_COMPRESS=()
+    FILES_TO_COMPRESS+=("Microsoft.SqlServer.CSharpExtension.runtimeconfig.json")
+    FILES_TO_COMPRESS+=("Microsoft.SqlServer.CSharpExtension.deps.json")
+
+    # Add all .dll and .so files
+    for f in "$BUILD_OUTPUT"/*.dll "$BUILD_OUTPUT"/*.so; do
+        [ -e "$f" ] && FILES_TO_COMPRESS+=("$(basename "$f")")
+    done
+
+    # Include .pdb files for debug builds
+    if [ "$BUILD_CONFIGURATION" = "debug" ]; then
+        for f in "$BUILD_OUTPUT"/*.pdb; do
+            [ -e "$f" ] && FILES_TO_COMPRESS+=("$(basename "$f")")
+        done
+    fi
+
+    # Include the shared/ directory (framework file copies for component hosting).
+    # The build script creates shared/Microsoft.NETCore.App/<version>/ with copies
+    # of the root DLLs/SOs so hostfxr can resolve the bundled .NET runtime.
+    # File copies are used because SQL Server's archive extraction does not preserve
+    # symlinks or hard links.
+    if [ -d "$BUILD_OUTPUT/shared" ]; then
+        FILES_TO_COMPRESS+=("shared")
+    fi
+
+    # Package the binaries.
+    #
+    # IMPORTANT: SQL Server on Linux extracts CREATE EXTERNAL LANGUAGE archives
+    # using its built-in zip code (the same path R/Python/Java use). Tar.gz is
+    # NOT supported on Linux - exthost would silently fail to find the .so and
+    # exit with status 9 immediately after launch. Use zip + .zip extension to
+    # match the format expected by the satellite/launchpad.
+    #
+    # Both .tar.gz and .zip outputs are produced for backward compatibility:
+    # - dotnet-core-CSharp-lang-extension-linux.zip   (NEW - used by Linux launchpad)
+    # - dotnet-core-CSharp-lang-extension.tar.gz      (legacy/Windows-style, kept for compat)
+    pushd "$BUILD_OUTPUT" > /dev/null
+
+    # Primary: zip archive (Linux SQL Server expects this format)
+    rm -f "$BUILD_OUTPUT/packages/dotnet-core-CSharp-lang-extension-linux.zip"
+    zip -r "$BUILD_OUTPUT/packages/dotnet-core-CSharp-lang-extension-linux.zip" "${FILES_TO_COMPRESS[@]}" > /dev/null
+    check_error $? "Failed to create zip for dotnet-core-CSharp-extension for configuration=$BUILD_CONFIGURATION"
+
+    # Legacy: tar.gz (kept for any callers still referencing the old name)
+    tar -czf "$BUILD_OUTPUT/packages/dotnet-core-CSharp-lang-extension.tar.gz" "${FILES_TO_COMPRESS[@]}"
+    check_error $? "Failed to create tarball for dotnet-core-CSharp-extension for configuration=$BUILD_CONFIGURATION"
+    popd > /dev/null
+
+    echo "Success: Compressed dotnet-core-CSharp-extension for $BUILD_CONFIGURATION configuration."
+done