python3 fails with "Operation not permitted" (exit 126) on arm64 (Raspberry Pi 5) — confirmed not host-related

[juliovgoncalves]

What installation are you running?

Production (netalertx) 📦

Is there an existing issue for this?

• I have searched the existing open and closed issues and I checked the docs https://docs.netalertx.com/

The issue occurs in the following browsers. Select at least 2.

• Firefox
• Chrome
• Edge
• Safari (unsupported) - PRs welcome
• N/A - This is an issue with the backend

Current Behavior

Environment: Raspberry Pi 5, Debian 12 (Bookworm), aarch64, Docker 29.6.1, containerd 2.2.5, runc 1.3.6, overlay2/extfs storage
Issue: Container fails at /entrypoint.d/15-mounts.py with env: can't execute 'python3': Operation not permitted, on both :latest and :26.3.7.
Ruled out (with evidence):

Privilege drop / PUID — fails identically running fully as root (PUID=0)
AppArmor — disabled at kernel level (aa-status: not mounted)
Seccomp — fails identically with --security-opt seccomp=unconfined
Rootless Docker — confirmed standard rootful daemon, cgroup v2/systemd
Docker storage — overlay2/extfs, no noexec, confirmed via findmnt
binfmt_misc — disabled the host's python3.11 handler, no change
systemd hardening on docker/containerd — fully open (no NoNewPrivileges, full CapabilityBoundingSet)
Host-wide exec — ruled out entirely: python:3.12-slim and alpine:latest both run perfectly on the same host

Conclusion: appears isolated to the netalertx image's bundled Python binary/build itself.

Expected Behavior

not failing to start

Steps To Reproduce

No response

Relevant app.conf settings

docker-compose.yml

Debug or Trace enabled

• I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.

Relevant app.log section

  PASTE LOG HERE. Using the triple backticks preserves format.

Docker Logs

  PASTE DOCKER LOG HERE. Using the triple backticks preserves format.

[jokob-sk]

Hi there,

Thanks for the report. Unfortunately I don't have an Raspberry pi 5 to test, but I have a Raspberry Pi 4 where I tested this out just now. Unfortunately, I can't seem to reproduce the error there. It seems to be running fine so far:

Linux raspberrypi 6.12.47+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.12.47-1+rpt1 (2025-09-16) aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jul  1 19:56:46 2026 from 192.168.1.82

jokob@raspberrypi:~ $ docker info
Client: Docker Engine - Community
 Version:    29.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.30.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.40.3
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  model: Docker Model Runner (Docker Inc.)
    Version:  v1.0.1
    Path:     /usr/libexec/docker/cli-plugins/docker-model

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 29.0.2
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: fcd43222d6b07379a4be9786bda52438f0dd16a1
 runc version: v1.3.3-0-gd842d771
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.47+rpt-rpi-v8
 Operating System: Debian GNU/Linux 13 (trixie)
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.708GiB
 Name: raspberrypi
 ID: be0e7b34-1697-497b-90e1-287a7abdbafe
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Firewall Backend: iptables

docker-compose:

services:
  netalertx:
  #use an environmental variable to set host networking mode if needed
    container_name: netalertx                       # The name when you docker contiainer ls
    image: ghcr.io/netalertx/netalertx:latest
    network_mode: ${NETALERTX_NETWORK_MODE:-host}   # Use host networking for ARP scanning and other services

    read_only: true                                 # Make the container filesystem read-only
    cap_drop:                                       # Drop all capabilities for enhanced security
      - ALL
    cap_add:                                        # Add only the necessary capabilities
      - NET_ADMIN                                   # Required for ARP scanning
      - NET_RAW                                     # Required for raw socket operations
      - NET_BIND_SERVICE                            # Required to bind to privileged ports (nbtscan)
      - CHOWN                                       # Required for root-entrypoint to chown /data + /tmp before dropping privileges
      - SETUID                                      # Required for root-entrypoint to switch to non-root user
      - SETGID                                      # Required for root-entrypoint to switch to non-root group
    # --- ARP FLUX MITIGATION ---
    # Note: When using `network_mode: host`, these sysctls require the
    # NET_ADMIN capability to be applied to the host namespace.
    # 
    # If your environment restricts capabilities, or you prefer to configure
    # them on the Host OS, REMOVE the sysctls block below and apply via:
    # sudo sysctl -w net.ipv4.conf.all.arp_ignore=1 net.ipv4.conf.all.arp_announce=2
    # ---------------------------
#    sysctls:                                        # ARP flux mitigation (reduces duplicate/ambiguous ARP behavior on host networking)
#      net.ipv4.conf.all.arp_ignore: 1
 #     net.ipv4.conf.all.arp_announce: 2

    volumes:
      - type: volume                                # Persistent Docker-managed named volume for config + database
        source: netalertx_data
        target: /data                               # `/data/config` and `/data/db` live inside this mount
        read_only: false

    # Example custom local folder called /home/user/netalertx_data
    # - type: bind
    #   source: /home/user/netalertx_data
    #   target: /data
    #   read_only: false
    # ... or use the alternative format
    # - /home/user/netalertx_data:/data:rw

      - type: bind                                  # Bind mount for timezone consistency
        source: /etc/localtime
        target: /etc/localtime
        read_only: true

      # Mount your DHCP server file into NetAlertX for a plugin to access
      # - path/on/host/to/dhcp.file:/resources/dhcp.file

    # tmpfs mount consolidates writable state for a read-only container and improves performance
    # uid/gid default to the service user (NETALERTX_UID/GID, default 20211)
    # mode=1700 grants rwx------ permissions to the runtime user only
    tmpfs:
      # Comment out to retain logs between container restarts - this has a server performance impact.
      - "/tmp:uid=${NETALERTX_UID:-20211},gid=${NETALERTX_GID:-20211},mode=1700,rw,noexec,nosuid,nodev,async,noatime,nodiratime"

      # Retain logs - comment out tmpfs /tmp if you want to retain logs between container restarts
      # Please note if you remove the /tmp mount, you must create and maintain sub-folder mounts.
      # - /path/on/host/log:/tmp/log
      # - "/tmp/api:uid=${NETALERTX_UID:-20211},gid=${NETALERTX_GID:-20211},mode=1700,rw,noexec,nosuid,nodev,async,noatime,nodiratime"
      # - "/tmp/nginx:uid=${NETALERTX_UID:-20211},gid=${NETALERTX_GID:-20211},mode=1700,rw,noexec,nosuid,nodev,async,noatime,nodiratime"
      # - "/tmp/run:uid=${NETALERTX_UID:-20211},gid=${NETALERTX_GID:-20211},mode=1700,rw,noexec,nosuid,nodev,async,noatime,nodiratime"

    environment:
      LISTEN_ADDR: ${LISTEN_ADDR:-0.0.0.0}                   # Listen for connections on all interfaces
      PORT: ${PORT:-20211}                                   # Application port
      GRAPHQL_PORT: ${GRAPHQL_PORT:-20212}                   # GraphQL API port (passed into APP_CONF_OVERRIDE at runtime)
  #    NETALERTX_DEBUG: ${NETALERTX_DEBUG:-0}                 # 0=kill all services and restart if any dies. 1 keeps running dead services.
  #    PUID: 20211                             # Runtime PUID override, set to 0 to run as root
  #    PGID: 20211                             # Runtime PGID override

    # Resource limits to prevent resource exhaustion
    mem_limit: 2048m            # Maximum memory usage
    mem_reservation: 1024m      # Soft memory limit
    cpu_shares: 512             # Relative CPU weight for CPU contention scenarios
    pids_limit: 512             # Limit the number of processes/threads to prevent fork bombs
    logging:
      options:
        max-size: "10m"         # Rotate log files after they reach 10MB
        max-file: "3"           # Keep a maximum of 3 log files

    # Always restart the container unless explicitly stopped
    restart: unless-stopped

volumes:                        # Persistent volume for configuration and database storage
  netalertx_data:

jokob@raspberrypi:~/Docker/netalertx $ docker compose up
[+] Running 1/1
 ✔ Container netalertx  Recreated                                                                                                                                                                   1.6s 
Attaching to netalertx
netalertx  | 
netalertx  |  _   _      _    ___  _           _  __   __
netalertx  | | \ | |    | |  / _ \| |         | | \ \ / /
netalertx  | |  \| | ___| |_/ /_\ \ | ___ _ __| |_ \ V /
netalertx  | | .   |/ _ \ __|  _  | |/ _ \  __| __|/   \
netalertx  | | |\  |  __/ |_| | | | |  __/ |  | |_/ /^\ \
netalertx  | \_| \_/\___|\__\_| |_/_|\___|_|   \__\/   \/
netalertx  |    Network intruder and presence detector.
netalertx  |    https://netalertx.com
netalertx  | 
netalertx  | 
netalertx  | Startup pre-checks
netalertx  | --> data migration.sh 
netalertx  | --> capabilities audit.sh 
netalertx  | --> mounts.py 
netalertx  |  Path                     | R | W | Mount | RAMDisk | Performance | DataLoss 
netalertx  | --------------------------+---+---+-------+---------+-------------+----------
netalertx  |  /data                    | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /data/db                 | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /data/config             | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /tmp/run/tmp             | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/api                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/log                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/run                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/nginx/active-config | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  | --> first run config.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | 🆕  First run detected. Default configuration written to /data/config/app.conf.
netalertx  | 
netalertx  |     Review your settings in the UI or edit the file directly before trusting
netalertx  |     this instance in production.
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> first run db.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | 🆕  First run detected — building initial database at: /data/db/app.db
netalertx  | 
netalertx  |     Do not interrupt this step. When complete, consider backing up the fresh
netalertx  |     DB before onboarding sensitive or critical networks.
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> mandatory folders.sh 
netalertx  |     * Creating NetAlertX log directory.
netalertx  |     * Creating NetAlertX API cache.
netalertx  |     * Creating System services runtime directory.
netalertx  |     * Creating nginx active configuration directory.
netalertx  |     * Creating Plugins log.
netalertx  |     * Creating System services run log.
netalertx  |     * Creating System services run tmp.
netalertx  |     * Creating DB locked log.
netalertx  |     * Creating Execution queue log.
netalertx  | --> apply conf override.sh 
netalertx  | --> override individual settings.sh 
netalertx  | --> host optimization.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | ⚠️  WARNING: ARP flux sysctls are not set.
netalertx  | 
netalertx  |     Expected values:
netalertx  |       net.ipv4.conf.all.arp_ignore=1
netalertx  |       net.ipv4.conf.all.arp_announce=2
netalertx  | 
netalertx  |     Note: If using 'network_mode: host', setting these via docker-compose sysctls
netalertx  |     requires the NET_ADMIN capability. When granted, these sysctls will
netalertx  |     modify the host namespace. Otherwise, you must configure them directly
netalertx  |     on your host operating system instead.
netalertx  | 
netalertx  |     Detection accuracy may be reduced until configured.
netalertx  | 
netalertx  |     See: https://docs.netalertx.com/docker-troubleshooting/arp-flux-sysctls/
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> writable config.sh 
netalertx  | --> nginx config.sh 
netalertx  | --> expected user id match.sh 
netalertx  | --> host mode network.sh 
netalertx  | --> excessive capabilities.sh 
netalertx  | --> appliance integrity.sh 
netalertx  | --> ports available.sh 
netalertx  | APP_CONF_OVERRIDE detected (set from GRAPHQL_PORT)
netalertx  | Starting /usr/sbin/php-fpm83 -y "/services/config/php/php-fpm.conf" -F (tee stderr to app.php_errors.log)
netalertx  | Starting supercronic --quiet "/services/config/cron/crontab" >>"/tmp/log/cron.log" 2>&1 &
netalertx  | Starting python3  -m server > /tmp/log/stdout.log 2> >(tee /tmp/log/stderr.log >&2)
netalertx  | Starting /usr/sbin/nginx -p "/tmp/run/" -c "/tmp/nginx/active-config/nginx.conf" -g "error_log stderr; error_log /tmp/log/nginx-error.log; daemon off;" &
netalertx  | Successfully updated IEEE OUI database (114789 entries)

[jokob-sk]

If it's practical, could you try upgrading the host to Debian 13 (Trixie), or test on another Debian 13 system? I can't promise it'll fix the issue, but since the current image runs correctly on my Debian 13 Raspberry Pi 4, it would help narrow down whether this is an OS/kernel/runtime-specific problem.

There is also a similar issue reported in #1648 and #1545 with the same env: can't execute 'python3': Operation not permitted error. Unfortunately, I haven't been able to identify a common root cause, and I can't reproduce it on my Raspberry Pi 4 (aarch64), where the current image starts normally.

Three similar reports over more than a year suggest it's a rare environment-specific interaction rather than a generally broken image. It would be still great to figure out the root cause.

[juliovgoncalves]

this is my setup:
Environment: Raspberry Pi 5, Debian 12 (Bookworm), aarch64, Docker 29.6.1, containerd 2.2.5, runc 1.3.6, overlay2/extfs storage

[jokob-sk]

Thanks. That confirms you're running Raspberry Pi 5 on Debian 12 (Bookworm) with a 64-bit userspace. Unfortunately I still can't identify a common factor. The current image starts normally on my Raspberry Pi 4 (aarch64), so there appears to be an environment-specific interaction that I haven't been able to reproduce yet.

If it's practical, could you try upgrading the host to Debian 13 (Trixie), or test on another Debian 13 system? I can't promise it'll fix the issue, but since the current image runs correctly on my Debian 13 Raspberry Pi 4, it would help narrow down whether this is an OS/kernel/runtime-specific problem.