Add an option to warn when C extensions are loaded

Author: timfel

CHANGELOG.md

@@ -7,6 +7,7 @@ language runtime. The main focus is on user-observable behavior of the engine.
 * Updated developer metadata of Maven artifacts.
 
 ## Version 24.1.0
+* GraalPy is now considered stable for pure Python workloads. While many workloads involving native extension modules work, we continue to consider them experimental. You can use the command-line option `--python.WarnExperimentalFeatures` to enable warnings for such modules at runtime. In Java embeddings, set the context option `python.WarnExperimentalFeatures` to true.
 * Update to Python 3.11.7
 * We now provide intrinsified `_pickle` module also in the community version.
 * `polyglot.eval` now raises more meaningful exceptions. Unavaliable languages raise `ValueError`. Exceptions from the polyglot language are raised directly as interop objects (typed as `polyglot.ForeignException`). The shortcut for executing python files without specifying language has been removed, use regular `eval` for executing Python code.

README.md

@@ -6,6 +6,7 @@
 
 GraalPy is a high-performance implementation of the [Python](https://www.python.org/) language for the JVM built on [GraalVM](https://www.graalvm.org/).
 GraalPy has first-class support for embedding in Java and can turn Python applications into fast, standalone binaries.
+GraalPy is ready for production running pure Python code and has experimental support for many popular native extension modules.
 
 ## Why GraalPy?
 
@@ -17,15 +18,15 @@ GraalPy has first-class support for embedding in Java and can turn Python applic
 
 **Compatible with the Python ecosystem**
 
-* Install [packages](docs/user/Python-Runtime.md#installing-packages) like *NumPy*, *PyTorch*, or *Tensorflow*; run [Hugging Face](https://huggingface.co/) models like *Stable Diffusion* or *GPT*
-* See if the packages you need work with our [Python Compatibility Checker](https://www.graalvm.org/python/compatibility/)
 * Use almost any standard Python feature, the CPython tests run on every commit and pass ~85%
 ![](docs/user/assets/mcd.svg#gh-light-mode-only)![](docs/user/assets/mcd-dark.svg#gh-dark-mode-only)<sup>
 We run the tests of the [most depended on PyPI packages](https://libraries.io/pypi) every day.
 For 96% of those packages a recent version can be installed on GraalPy and GraalPy passes about 50% of all tests of all packages combined.
 We assume that CPython not passing 100% of all tests is due to problems in our infrastructure that may also affect GraalPy.
 Packages where CPython fails all tests are marked as "not tested" for both CPython and GraalPy.
 </sup>
+* See if the packages you need work according to our [Python Compatibility Checker](https://www.graalvm.org/python/compatibility/)
+* Support for native extension modules is considered experimental, but you can already install [packages](docs/user/Python-Runtime.md#installing-packages) like *NumPy*, *PyTorch*, or *Tensorflow*; run [Hugging Face](https://huggingface.co/) models like *Stable Diffusion* or *GPT*
 
 **Runs Python code faster**
 

graalpython/com.oracle.graal.python/src/com/oracle/graal/python/builtins/objects/cext/common/CExtContext.java

@@ -53,6 +53,7 @@
 
 import java.io.IOException;
 import java.nio.file.LinkOption;
+import java.util.Set;
 
 import org.graalvm.collections.Pair;
 import org.graalvm.shadowed.com.ibm.icu.impl.Punycode;
@@ -274,6 +275,15 @@ private static String dlopenFlagsToString(int flags) {
         return str;
     }
 
+    private static final Set<String> C_EXT_SUPPORTED_LIST = Set.of(
+                    // Stdlib modules are considered supported
+                    "_cpython_sre",
+                    "_cpython_unicodedata",
+                    "_sha3",
+                    "_sqlite3",
+                    "termios",
+                    "pyexpat");
+
     /**
      * This method loads a C extension module (C API) and will initialize the corresponding native
      * contexts if necessary.
@@ -295,6 +305,10 @@ private static String dlopenFlagsToString(int flags) {
     public static Object loadCExtModule(Node location, PythonContext context, ModuleSpec spec, CheckFunctionResultNode checkFunctionResultNode)
                     throws IOException, ApiInitException, ImportException {
 
+        if (context.getOption(PythonOptions.WarnExperimentalFeatures) && !C_EXT_SUPPORTED_LIST.contains(spec.name.toJavaStringUncached())) {
+            getLogger().warning(() -> "Loading C extension module %s from '%s'. Support for the Python C API is considered experimental.".formatted(spec.name, spec.path));
+        }
+
         // we always need to load the CPython C API
         CApiContext cApiContext = CApiContext.ensureCapiWasLoaded(location, context, spec.name, spec.path);
         Object library;

graalpython/com.oracle.graal.python/src/com/oracle/graal/python/runtime/PythonOptions.java

@@ -388,6 +388,9 @@ private PythonOptions() {
     @EngineOption @Option(category = OptionCategory.INTERNAL, usageSyntax = "true|false", help = "If true, uses native storage strategy for primitive types") //
     public static final OptionKey<Boolean> UseNativePrimitiveStorageStrategy = new OptionKey<>(false);
 
+    @Option(category = OptionCategory.EXPERT, usageSyntax = "true|false", help = "Print warnings when using experimental features at runtime.", stability = OptionStability.STABLE) //
+    public static final OptionKey<Boolean> WarnExperimentalFeatures = new OptionKey<>(false);
+
     public static final OptionDescriptors DESCRIPTORS = new PythonOptionsOptionDescriptors();
 
     @CompilationFinal(dimensions = 1) private static final OptionKey<?>[] ENGINE_OPTION_KEYS;