libs/libc/elf: Load FDPIC modules through the ELF loader

Documentation/components/index.rst

@@ -14,6 +14,7 @@ case, you can head to the :doc:`reference <../reference/index>`.
    binfmt.rst
    concurrency/index.rst
    drivers/index.rst
+   fdpic.rst
    nxflat.rst
    nxgraphics/index.rst
    paging.rst

Documentation/components/tools/fdpic.rst

@@ -0,0 +1,80 @@
+=====================================
+fdpic: FDPIC module build tooling
+=====================================
+
+``tools/fdpic`` is everything needed to build an FDPIC module out of tree.  A
+module is an ELF shared object whose read-only segment the target maps straight
+out of flash and executes in place, while its writable segment is copied to RAM
+once per running instance.  It links against nothing: libc and everything else
+are imported from the firmware's exported symbol table at load time.
+
+The loader that consumes these modules is the ELF loader, enabled by
+``CONFIG_FDPIC``.  The format, the toolchain and the load-time contract are
+described in :doc:`/components/fdpic`.
+
+Contents
+========
+
+============================ ==================================================
+File                         Purpose
+============================ ==================================================
+``nuttx-fdpic.mk``           The module build itself; include it from a
+                             two-line makefile
+``fdpic-verify.sh``          Checks a built module's imports resolve against
+                             the firmware
+``nuttx-exports.sh``         Turns ``libs/libc/exec_symtab.c`` into a symbol
+                             list
+``fdpic-embed.py``           Turns a built module into a C header, for carrying
+                             one inside an image
+``build-binutils.sh``        Builds the ``arm-uclinuxfdpiceabi`` binutils, the
+                             one from-source dependency
+============================ ==================================================
+
+Building a module
+=================
+
+A whole module is three lines of makefile beside the source.  Taking
+``apps/examples/fdpicxip/modules/qsorter.c``, which is a module in its own
+right, as the source:
+
+.. code:: makefile
+
+   MODULE = qsorter
+   SRCS   = qsorter.c
+
+   include /path/to/nuttx/tools/fdpic/nuttx-fdpic.mk
+
+Then:
+
+.. code:: console
+
+   $ make NUTTX_DIR=/path/to/nuttx
+     CC   qsorter.c
+     LD   qsorter.fdpic
+   OK    qsorter.fdpic: FDPIC, entry 0x2a1, 4 imports resolved
+
+``NUTTX_DIR`` has to be a configured, built tree: the compile needs its headers
+and the verify step needs the export table generated into
+``libs/libc/exec_symtab.c``.
+
+Toolchain
+=========
+
+Two toolchains are involved.  The stock ``arm-none-eabi`` compiler does the
+compiling -- it emits perfectly good FDPIC objects for both C and C++ -- and
+``arm-uclinuxfdpiceabi`` **binutils** does the linking, because
+``arm-none-eabi-ld`` cannot produce an FDPIC object at all.  So the
+from-source dependency is binutils alone, which ``build-binutils.sh`` builds in
+about a minute.
+
+Verification runs as part of the default target on purpose: a module that
+imports a symbol the firmware does not export links perfectly happily and fails
+only once it is on the target, as a bare ``-ENOENT`` that names nothing.
+
+Modules carried inside an image
+===============================
+
+``fdpic-embed.py`` exists for applications that have to load a module before
+there is any way to put files on the target, so they embed one and write it out
+at run time.  ``apps/examples/fdpicxip/modules/`` uses it that way, and is the
+worked example of driving this tooling for several modules at once.

Documentation/platforms/arm/rp23xx/boards/pimoroni-pico-2-plus/index.rst

@@ -177,6 +177,21 @@ xipfs
 XIPFS mounted on the on-board flash, with the ``xipfs`` command and the
 XIPFS test suite.
 
+xipfs-fdpic
+-----------
+
+Same as ``xipfs``, plus the FDPIC module loader and the
+``fdpicxip`` demo, so the ``fdpic`` and ``reject`` sections of the XIPFS
+test suite have something to run.  The demo carries its modules as
+committed byte arrays, so nothing beyond the ordinary ARM toolchain is
+needed to build it; rebuilding those from source needs
+``arm-uclinuxfdpiceabi`` binutils.  See :doc:`/components/fdpic`.
+
+``CONFIG_ELF_STACKSIZE`` is 4096 here rather than the 2048 the rest of
+the board's tasks use.  A module that calls into the firmware's printf
+family overflows 2048, and with no MPU that is a lockup rather than a
+diagnostic.
+
 xipfs-nxflat
 ------------
 

arch/Kconfig

@@ -692,6 +692,13 @@ config ARCH_HAVE_ELF_EXECUTABLE
 	bool
 	default n
 
+config ARCH_HAVE_ELF_FDPIC
+	bool
+	default n
+	---help---
+		The architecture has a PIC base register and the ELF relocations
+		that an FDPIC object uses.
+
 config ARCH_HAVE_TRUSTZONE
 	bool
 	default n

arch/arm/Kconfig

@@ -1061,6 +1061,7 @@ config ARCH_ARMV7M
 	default n
 	select ARCH_HAVE_CPUINFO
 	select ARCH_HAVE_DEBUG
+	select ARCH_HAVE_ELF_FDPIC
 	select ARCH_HAVE_PERF_EVENTS
 
 config ARCH_CORTEXM3
@@ -1210,6 +1211,7 @@ config ARCH_ARMV8M
 	default n
 	select ARCH_HAVE_CPUINFO
 	select ARCH_HAVE_DEBUG
+	select ARCH_HAVE_ELF_FDPIC
 	select ARCH_HAVE_PERF_EVENTS
 
 config ARCH_CORTEXM23

arch/arm/include/arch.h

@@ -79,6 +79,56 @@ do { \
   ); \
 } while (0)
 
+#ifdef CONFIG_FDPIC
+
+/****************************************************************************
+ * Name: up_fdpic_invoke
+ *
+ * Description:
+ *   Call a module entry point with the module data base in the PIC base
+ *   register.  Put the caller data base back after the call.
+ *
+ *   up_setpicbase() cannot do this.  The register must hold the module base
+ *   for one call only, and C cannot tell the compiler that the register is
+ *   live across that call.  Thus the save, the install, the branch and the
+ *   restore must be one sequence.
+ *
+ *   The sequence is safe against a context switch or an interrupt.  The
+ *   register is REG_PIC in the saved context, and the base firmware keeps
+ *   the register reserved.
+ *
+ * Input Parameters:
+ *   entry - The code address to enter.
+ *   arg   - The one word argument, passed in r0.
+ *   got   - The module data base to install.
+ *
+ ****************************************************************************/
+
+static inline void up_fdpic_invoke(uintptr_t entry, uintptr_t arg,
+                                   uintptr_t got)
+{
+  register uintptr_t r0v __asm__ ("r0") = arg;
+
+  /* arg stays in r0, which is the first argument and the scratch of the
+   * call.  Thus the asm needs registers for entry and got only.  The PIC
+   * register goes on the stack, and r4 goes with it to keep the push
+   * aligned to 8 bytes.
+   */
+
+  __asm__ __volatile__
+  (
+    "push {r4, " PIC_REG_STRING "}\n"   /* Save the caller's base       */
+    "mov " PIC_REG_STRING ", %[got]\n"  /* Install the module's base    */
+    "blx %[entry]\n"                    /* Enter the module             */
+    "pop {r4, " PIC_REG_STRING "}\n"    /* Restore the caller's base    */
+    : "+r" (r0v)
+    : [entry] "r" (entry), [got] "r" (got)
+    : "r1", "r2", "r3", "r12", "lr", "cc", "memory"
+  );
+}
+
+#endif /* CONFIG_FDPIC */
+
 #endif /* CONFIG_PIC */
 
 #ifdef CONFIG_ARCH_ADDRENV

arch/arm/include/elf.h

@@ -206,6 +206,24 @@
 #define R_ARM_THM_TLS_DESCSEQ16  129           /* Thumb16 */
 #define R_ARM_THM_TLS_DESCSEQ32  130           /* Thumb32 */
 
+/* FDPIC relocations.
+ *
+ * Under the FDPIC ABI each PT_LOAD segment is placed independently, so a
+ * function pointer cannot be a bare code address: it has to carry the data
+ * base its callee will need.  A "function descriptor" is that pair, and
+ * these relocations are how the loader is asked to build and reference
+ * them.  Values are from the ARM FDPIC ABI as implemented by binutils
+ * (include/elf/arm.h).
+ */
+
+#define R_ARM_GOTFUNCDESC        161           /* Data      GOT entry holding a descriptor */
+#define R_ARM_GOTOFFFUNCDESC     162           /* Data      GOT-relative descriptor */
+#define R_ARM_FUNCDESC           163           /* Data      Address of a descriptor */
+#define R_ARM_FUNCDESC_VALUE     164           /* Data      The descriptor itself: {code, GOT} */
+#define R_ARM_TLS_GD32_FDPIC     165           /* Data */
+#define R_ARM_TLS_LDM32_FDPIC    166           /* Data */
+#define R_ARM_TLS_IE32_FDPIC     167           /* Data */
+
 /* Processor specific values for the Phdr p_type field.  */
 
 #define PT_ARM_EXIDX             (PT_LOPROC + 1) /* ARM unwind segment.  */
@@ -247,10 +265,72 @@
 #define DT_ARM_PREEMPTMAP        0x70000002
 #define DT_ARM_RESERVED2         0x70000003
 
+/* Loader state the FDPIC relocations need but a relocation cannot carry:
+ * the object's data base, and a descriptor pool cursor that has to survive
+ * from one relocation to the next.
+ */
+
+/* The relocations that only an FDPIC object may use.  Seeing one in an
+ * object whose OS/ABI byte does not say FDPIC means the marker was lost.
+ */
+
+#define ARCH_ELF_RELOC_ISFDPIC(t)                    \
+  ((t) == R_ARM_FUNCDESC || (t) == R_ARM_FUNCDESC_VALUE)
+
+#define ARCH_ELFDATA             1
+
+#define ARCH_ELFDATA_INIT(d, l)          \
+  do                                     \
+    {                                    \
+      (d)->fdpic    = (l)->fdpic;        \
+      (d)->gotaddr  = (l)->gotaddr;      \
+      (d)->descpool = (l)->descpool;     \
+      (d)->ndesc    = (l)->ndesc;        \
+      (d)->usedesc  = (l)->usedesc;      \
+    }                                    \
+  while (0)
+
+#define ARCH_ELFDATA_FINI(d, l)          \
+  do                                     \
+    {                                    \
+      (l)->usedesc = (d)->usedesc;       \
+    }                                    \
+  while (0)
+
 /****************************************************************************
  * Public Types
  ****************************************************************************/
 
+#ifndef __ASSEMBLY__
+
+/* An FDPIC function pointer: the code, plus the data base to enter it
+ * with.
+ */
+
+struct arm_fdpic_desc_s
+{
+  uintptr_t entry;         /* Address of the code */
+  uintptr_t got;           /* Data base to install before branching */
+};
+
+struct arch_elfdata_s
+{
+  uint8_t   fdpic;         /* The object is an FDPIC one */
+  uintptr_t gotaddr;       /* DT_PLTGOT: this object's data base */
+  uintptr_t descpool;      /* Base of the descriptor pool */
+  uint16_t  ndesc;         /* Capacity, in descriptors */
+  uint16_t  usedesc;       /* Next free slot */
+  uint8_t   symisdesc;     /* Symbol value is a descriptor, not code */
+  uint8_t   pltrel;        /* Relocation comes from DT_JMPREL, so the word
+                            * it overwrites is a lazy binding stub and not
+                            * an addend
+                            */
+};
+
+typedef struct arch_elfdata_s arch_elfdata_t;
+
+#endif /* __ASSEMBLY__ */
+
 typedef struct __EIT_entry
 {
   unsigned long fnoffset;

binfmt/Kconfig

@@ -60,6 +60,39 @@ config ELF_STACKSIZE
 	default DEFAULT_TASK_STACKSIZE
 	---help---
 		This is the default stack size that will be used when starting ELF binaries.
+
+config FDPIC
+	bool "FDPIC modules"
+	default n
+	select PIC
+	depends on ARCH_HAVE_ELF_FDPIC
+	depends on BUILD_FLAT
+	---help---
+		Load ELF modules built for the FDPIC ABI.
+
+		An FDPIC module places its read-only and writable segments
+		independently, so its text can be executed directly out of flash
+		while only the writable segment is copied to RAM, once per running
+		instance.  A filesystem that can show its media, such as XIPFS or
+		ROMFS, gives that result.  On any other filesystem the loader copies
+		the text to RAM, and the module runs but shares nothing.
+
+		Building a module needs an arm-uclinuxfdpiceabi linker.  The stock
+		arm-none-eabi compiler emits correct FDPIC objects for both C and
+		C++, so only the link needs it.
+
+		What this adds over the position independent ELF support already
+		present is a function pointer that carries its own data base, as a
+		two word descriptor rather than a bare code address.  That is what
+		lets a module be called back on a thread it did not create, such as
+		the work queue worker that runs a SIGEV_THREAD notification.
+
+		Selecting this makes ten libc and sched entry points that can
+		accept a callback from a module resolve such a descriptor before
+		storing or branching to it.  Each costs a register read and a
+		branch on a path that is not hot.
+
+		FDPIC is specified only for ARM Thumb-2.
 endif
 endif