From 354d29a63a86ceb5113b0cd3963414e8b7d9e10d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1s=20Gr=C3=BCner?= <47506558+MegaRedHand@users.noreply.github.com> Date: Mon, 8 Jun 2026 10:51:45 -0300 Subject: [PATCH 1/2] feat: Shadow simulator compatibility behind shadow-integration feature Adds opt-in support for running ethlambda under the Shadow network simulator (for the lean-shadow-fuzzer), gated so the default build, main, and the committed Cargo.lock are unchanged. jemalloc: Jemalloc causes programs to deadlock during process startup under Shadow (https://github.com/shadow/shadow/issues/3763). The `jemalloc` feature (default-on) pulls the optional `tikv-jemallocator` dependency; the Shadow binary builds with `--no-default-features` to drop it entirely. A `compile_error!` catches enabling `shadow-integration` without `--no-default-features` (which would leave jemalloc linked). `jemalloc_pprof` stays: its `PROF_CTL` is `None` without the allocator, so the heap-profiling endpoints degrade gracefully. runtime: Shadow single-steps execution, so a multi-threaded runtime only adds scheduling noise. The `current_thread` flavor is gated behind `shadow-integration`. This is an optimization, not a requirement. quinn-udp: Shadow's UDP emulation lacks GSO/GRO batch syscalls, so quinn-udp must fall back to plain send_to/recv_from (see `shadow/quinn-udp-patch`). A Cargo `[patch]` cannot be feature-gated, so it is kept OUT of the committed manifest and injected at build time by `shadow/build.sh` (and the Dockerfile's SHADOW path). The committed Cargo.lock therefore stays identical to main. Adds `make shadow-build` / `make shadow-docker-build`, the `shadow/` directory (patch crate, injection script, README), plus `SHADOW`, `NO_DEFAULT_FEATURES` and `LOCKED` Docker build args. Ref: kamilsa/ethlambda@ed5a447 --- .dockerignore | 1 + Cargo.toml | 5 + Dockerfile | 20 +++- Makefile | 16 ++- bin/ethlambda/Cargo.toml | 15 ++- bin/ethlambda/src/main.rs | 27 ++++- shadow/README.md | 82 ++++++++++++++ shadow/build.sh | 36 ++++++ shadow/cargo-patch.toml | 7 ++ shadow/quinn-udp-patch/Cargo.toml | 24 ++++ shadow/quinn-udp-patch/src/fallback.rs | 103 +++++++++++++++++ shadow/quinn-udp-patch/src/lib.rs | 150 +++++++++++++++++++++++++ 12 files changed, 476 insertions(+), 10 deletions(-) create mode 100644 shadow/README.md create mode 100755 shadow/build.sh create mode 100644 shadow/cargo-patch.toml create mode 100644 shadow/quinn-udp-patch/Cargo.toml create mode 100644 shadow/quinn-udp-patch/src/fallback.rs create mode 100644 shadow/quinn-udp-patch/src/lib.rs diff --git a/.dockerignore b/.dockerignore index d64551ab..835932ee 100644 --- a/.dockerignore +++ b/.dockerignore @@ -9,3 +9,4 @@ !LICENSE !.git/ !.cargo/ +!shadow/ diff --git a/Cargo.toml b/Cargo.toml index 5cc0cb47..ce3d21eb 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -77,3 +77,8 @@ eyre = "0.6" tikv-jemallocator = { version = "0.6", features = ["stats", "unprefixed_malloc_on_supported_platforms", "profiling"] } jemalloc_pprof = { version = "0.8", features = ["flamegraph"] } +# NOTE: Shadow-simulator builds also replace quinn-udp with the fallback crate +# in `shadow/quinn-udp-patch` (no GSO/GRO batch syscalls). A Cargo `[patch]` +# cannot be feature-gated, so it is NOT committed here; `shadow/build.sh` injects +# it into this manifest at build time. See `make shadow-build`. + diff --git a/Dockerfile b/Dockerfile index e425f249..976d8314 100644 --- a/Dockerfile +++ b/Dockerfile @@ -28,12 +28,28 @@ ENV BUILD_PROFILE=$BUILD_PROFILE ARG FEATURES="" ENV FEATURES=$FEATURES -RUN cargo chef cook --profile $BUILD_PROFILE --features "$FEATURES" --recipe-path recipe.json +# Disable default features (e.g. for `shadow-integration`, which must drop the +# default jemalloc allocator). Set to "--no-default-features" to enable. +ARG NO_DEFAULT_FEATURES="" +ENV NO_DEFAULT_FEATURES=$NO_DEFAULT_FEATURES + +# Cargo --locked by default. The Shadow build injects a [patch] (see below) that +# is absent from the committed lockfile, so it must build unlocked: set LOCKED= . +ARG LOCKED="--locked" +ENV LOCKED=$LOCKED + +RUN cargo chef cook --profile $BUILD_PROFILE $NO_DEFAULT_FEATURES --features "$FEATURES" --recipe-path recipe.json # Build application # Include .git so vergen-git2 can extract version info (branch, commit SHA) COPY --exclude=target . . -RUN cargo build --profile $BUILD_PROFILE --features "$FEATURES" --locked --bin ethlambda + +# Shadow builds inject the quinn-udp [patch] into the workspace manifest, since a +# Cargo patch cannot be feature-gated and is therefore not committed. Set SHADOW=1. +ARG SHADOW="" +RUN if [ -n "$SHADOW" ]; then cat shadow/cargo-patch.toml >> Cargo.toml; fi + +RUN cargo build --profile $BUILD_PROFILE $NO_DEFAULT_FEATURES --features "$FEATURES" $LOCKED --bin ethlambda # ARG is not resolved in COPY so we have to hack around it by copying the # binary to a temporary location diff --git a/Makefile b/Makefile index d1234c3f..bddf4b7a 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: help fmt lint docker-build run-devnet test docs docs-deps docs-serve +.PHONY: help fmt lint docker-build shadow-build shadow-docker-build run-devnet test docs docs-deps docs-serve help: ## 📚 Show help for each of the Makefile recipes @grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' @@ -24,6 +24,20 @@ docker-build: ## 🐳 Build the Docker image -t ghcr.io/lambdaclass/ethlambda:$(DOCKER_TAG) . @echo +shadow-build: ## 👻 Build a Shadow-simulator-compatible binary (single-threaded, no jemalloc) + ./shadow/build.sh cargo build --release --no-default-features --features shadow-integration --bin ethlambda + +shadow-docker-build: ## 👻🐳 Build a Shadow-compatible Docker image + docker build \ + --build-arg GIT_COMMIT=$(GIT_COMMIT) \ + --build-arg GIT_BRANCH=$(GIT_BRANCH) \ + --build-arg SHADOW=1 \ + --build-arg FEATURES=shadow-integration \ + --build-arg NO_DEFAULT_FEATURES=--no-default-features \ + --build-arg LOCKED= \ + -t ghcr.io/lambdaclass/ethlambda:$(DOCKER_TAG)-shadow . + @echo + # 2026-05-17 LEAN_SPEC_COMMIT_HASH:=f12000bd68a9640cffdfbd9a07503c9112d32bee diff --git a/bin/ethlambda/Cargo.toml b/bin/ethlambda/Cargo.toml index e5e22ee9..64d15dfe 100644 --- a/bin/ethlambda/Cargo.toml +++ b/bin/ethlambda/Cargo.toml @@ -5,6 +5,19 @@ edition.workspace = true authors.workspace = true license.workspace = true +[features] +default = ["jemalloc"] +# jemalloc allocator with heap profiling. Build the Shadow binary with +# `--no-default-features --features shadow-integration` to drop it: jemalloc's +# `unprefixed_malloc` overrides the system allocator symbols, conflicting with +# the Shadow simulator's preload shim. Cargo features are additive, so a feature +# cannot subtract a default-on dependency; hence the `--no-default-features`. +jemalloc = ["dep:tikv-jemallocator"] +# Shadow simulator compatibility: single-threaded tokio runtime and no jemalloc. +# The quinn-udp UDP fallback is a Cargo `[patch]` (which cannot be feature-gated), +# injected at build time by `shadow/build.sh` / `make shadow-build`. +shadow-integration = [] + [dependencies] ethlambda-blockchain.workspace = true ethlambda-network-api.workspace = true @@ -31,7 +44,7 @@ reqwest.workspace = true thiserror.workspace = true eyre.workspace = true -tikv-jemallocator.workspace = true +tikv-jemallocator = { workspace = true, optional = true } [build-dependencies] vergen-git2.workspace = true diff --git a/bin/ethlambda/src/main.rs b/bin/ethlambda/src/main.rs index d361b969..35aca5db 100644 --- a/bin/ethlambda/src/main.rs +++ b/bin/ethlambda/src/main.rs @@ -1,11 +1,21 @@ mod checkpoint_sync; mod version; -#[cfg(not(target_env = "msvc"))] +// Jemalloc causes programs to deadlock during process startup under Shadow. +// See https://github.com/shadow/shadow/issues/3763. Build the Shadow binary +// with `--no-default-features --features shadow-integration` to drop the +// (default) `jemalloc` feature and thus the `tikv-jemallocator` dependency. +#[cfg(all(feature = "jemalloc", feature = "shadow-integration"))] +compile_error!( + "the `jemalloc` feature is incompatible with `shadow-integration`; \ + build the Shadow binary with `--no-default-features --features shadow-integration`" +); + +#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))] #[global_allocator] static ALLOC: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc; -#[cfg(not(target_env = "msvc"))] +#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))] #[allow(non_upper_case_globals)] #[unsafe(export_name = "malloc_conf")] static malloc_conf: &[u8] = b"prof:true,prof_active:true,lg_prof_sample:19\0"; @@ -116,7 +126,12 @@ struct CliOptions { data_dir: PathBuf, } -#[tokio::main] +// Shadow single-steps execution in a discrete-event simulation, so the default +// multi-threaded runtime's worker threads add only scheduling noise, never +// parallelism. Use a single-threaded runtime under Shadow. This is an +// optimization, not a correctness requirement. +#[cfg_attr(not(feature = "shadow-integration"), tokio::main)] +#[cfg_attr(feature = "shadow-integration", tokio::main(flavor = "current_thread"))] async fn main() -> eyre::Result<()> { let filter = EnvFilter::builder() .with_default_directive(tracing::Level::INFO.into()) @@ -162,10 +177,10 @@ async fn main() -> eyre::Result<()> { })?; let p2p_socket = SocketAddr::new(IpAddr::from([0, 0, 0, 0]), options.gossipsub_port); - #[cfg(not(target_env = "msvc"))] + #[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))] info!("Using jemalloc allocator with heap profiling enabled"); - #[cfg(target_env = "msvc")] - info!("Using system allocator (MSVC target)"); + #[cfg(any(target_env = "msvc", not(feature = "jemalloc")))] + info!("Using system allocator"); info!(node_key=?options.node_key, "got node key"); diff --git a/shadow/README.md b/shadow/README.md new file mode 100644 index 00000000..efa0b7f9 --- /dev/null +++ b/shadow/README.md @@ -0,0 +1,82 @@ +# Shadow simulator integration + +Build support for running ethlambda under the [Shadow] network simulator, +used by the [lean-shadow-fuzzer]. Everything here is **opt-in**: a normal +`cargo build` / `make docker-build` is completely unaffected, and the committed +`Cargo.toml` / `Cargo.lock` are identical to a Shadow-free checkout. + +## Why Shadow needs special handling + +Shadow runs the real binary but emulates time, threads, and the network. Three +things in a stock ethlambda build need to change for Shadow: + +| Area | Stock build | Under Shadow | Why | +|------|-------------|--------------|-----| +| Allocator | jemalloc with `unprefixed_malloc` (interposes the global C `malloc`) | system allocator (drop jemalloc) | **Correctness.** Self-deadlocks at startup ([shadow#3763]): Shadow's shim `fopen`s `/proc/self/maps` on its first intercepted syscall, calling `malloc`, which re-enters jemalloc mid-init while it holds its non-recursive init lock. | +| QUIC UDP I/O | `quinn-udp` uses GSO/GRO batch syscalls (`sendmmsg`, segmentation offload) | fall back to `send_to`/`recv_from` | **Correctness.** Shadow's UDP emulation doesn't support those batch syscalls. | +| Tokio runtime | multi-threaded | `current_thread` | **Optimization only.** Shadow single-steps execution, so worker threads add only scheduling noise, never parallelism. | + +The allocator change (dropping jemalloc) and the runtime flavor are gated behind +the `shadow-integration` Cargo feature (jemalloc is dropped via +`--no-default-features`). The quinn-udp change is a Cargo `[patch]`, which +**cannot** be feature-gated, so it is injected into the manifest only at build +time (see below). Of the three, only the allocator and quinn-udp changes are +correctness requirements; the single-threaded runtime is purely a performance +choice. + +## Contents + +| Path | Purpose | +|------|---------| +| `quinn-udp-patch/` | Drop-in `quinn-udp` replacement (package name stays `quinn-udp`) routing every send/receive through plain `send_to`/`recv_from`, batch size 1 | +| `cargo-patch.toml` | The `[patch.crates-io]` snippet appended to the workspace `Cargo.toml` for Shadow builds | +| `build.sh` | Runs a cargo command with the patch temporarily injected, then restores `Cargo.toml` + `Cargo.lock` | + +## Building + +```bash +# Local binary (release, single-threaded, no jemalloc, quinn-udp fallback) +make shadow-build + +# Docker image, tagged ...:-shadow +make shadow-docker-build +``` + +`make shadow-build` is a thin wrapper around: + +```bash +./shadow/build.sh cargo build --release \ + --no-default-features --features shadow-integration --bin ethlambda +``` + +`shadow/build.sh` can run any cargo command against the patched workspace, e.g.: + +```bash +./shadow/build.sh cargo check --no-default-features --features shadow-integration +./shadow/build.sh cargo clippy --no-default-features --features shadow-integration +``` + +It backs up `Cargo.toml` and `Cargo.lock`, appends `cargo-patch.toml`, runs the +command, and restores both files on exit (even on failure), so the working tree +is left pristine. + +> [!NOTE] +> `--no-default-features` is required, not optional: Cargo features are additive, +> so the `shadow-integration` feature cannot *remove* the default `jemalloc` +> dependency on its own. A `compile_error!` in `bin/ethlambda/src/main.rs` fires +> if `shadow-integration` is built with jemalloc still enabled. + +## Docker + +`make shadow-docker-build` passes these build args to the standard `Dockerfile`: + +| Arg | Value | Effect | +|-----|-------|--------| +| `SHADOW` | `1` | Appends `shadow/cargo-patch.toml` to `Cargo.toml` before the build | +| `FEATURES` | `shadow-integration` | Enables the feature | +| `NO_DEFAULT_FEATURES` | `--no-default-features` | Drops jemalloc | +| `LOCKED` | (empty) | Builds unlocked, since the injected patch is absent from the committed lockfile | + +[Shadow]: https://shadow.github.io/ +[lean-shadow-fuzzer]: https://github.com/kamilsa/lean-shadow-fuzzer +[shadow#3763]: https://github.com/shadow/shadow/issues/3763 diff --git a/shadow/build.sh b/shadow/build.sh new file mode 100755 index 00000000..072cf353 --- /dev/null +++ b/shadow/build.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# +# Run a cargo command against the Shadow-patched workspace. +# +# A Cargo `[patch.crates-io]` table cannot be gated behind a feature flag, so the +# quinn-udp fallback patch is kept out of the committed Cargo.toml. This script +# temporarily appends it (from shadow/cargo-patch.toml), runs the given command, +# then restores Cargo.toml and Cargo.lock to their pristine state. +# +# Usage: +# shadow/build.sh cargo build --release \ +# --no-default-features --features shadow-integration --bin ethlambda +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT" + +if [ "$#" -eq 0 ]; then + echo "usage: $0 " >&2 + exit 64 +fi + +# Back up the manifest + lockfile so we can restore them verbatim, regardless of +# whether they were committed or had local edits. +cp Cargo.toml Cargo.toml.shadow-bak +cp Cargo.lock Cargo.lock.shadow-bak + +restore() { + mv -f Cargo.toml.shadow-bak Cargo.toml + mv -f Cargo.lock.shadow-bak Cargo.lock +} +trap restore EXIT + +cat shadow/cargo-patch.toml >> Cargo.toml + +"$@" diff --git a/shadow/cargo-patch.toml b/shadow/cargo-patch.toml new file mode 100644 index 00000000..c1e1b21a --- /dev/null +++ b/shadow/cargo-patch.toml @@ -0,0 +1,7 @@ + +# Shadow simulator compatibility: replace quinn-udp with a fallback that uses +# plain send_to/recv_from instead of GSO/GRO batch syscalls, which Shadow's UDP +# emulation does not support. Appended to the workspace Cargo.toml at build time +# by shadow/build.sh (and the Dockerfile's SHADOW path); never committed. +[patch.crates-io] +quinn-udp = { path = "shadow/quinn-udp-patch" } diff --git a/shadow/quinn-udp-patch/Cargo.toml b/shadow/quinn-udp-patch/Cargo.toml new file mode 100644 index 00000000..3d01eb2e --- /dev/null +++ b/shadow/quinn-udp-patch/Cargo.toml @@ -0,0 +1,24 @@ +[package] +name = "quinn-udp" +version = "0.5.14" +edition = "2021" + +[features] +default = ["tracing", "log"] +log = ["tracing/log"] +direct-log = ["dep:log"] +fast-apple-datapath = [] + +[dependencies] +libc = { version = "0.2" } +log = { version = "0.4", optional = true } +tracing = { version = "0.1", optional = true } + +[target.'cfg(not(all(target_family = "wasm", target_os = "unknown")))'.dependencies] +socket2 = { version = "0.5" } + +[build-dependencies] +cfg_aliases = { version = "0.2" } + +[lib] +bench = false diff --git a/shadow/quinn-udp-patch/src/fallback.rs b/shadow/quinn-udp-patch/src/fallback.rs new file mode 100644 index 00000000..56d0b26e --- /dev/null +++ b/shadow/quinn-udp-patch/src/fallback.rs @@ -0,0 +1,103 @@ +use std::{ + io::{self, IoSliceMut}, + sync::Mutex, + time::Instant, +}; + +use super::{IO_ERROR_LOG_INTERVAL, RecvMeta, Transmit, UdpSockRef, log_sendmsg_error}; + +#[derive(Debug)] +pub struct UdpSocketState { + last_send_error: Mutex, +} + +impl UdpSocketState { + pub fn new(socket: UdpSockRef<'_>) -> io::Result { + socket.0.set_nonblocking(true)?; + let now = Instant::now(); + Ok(Self { + last_send_error: Mutex::new(now.checked_sub(2 * IO_ERROR_LOG_INTERVAL).unwrap_or(now)), + }) + } + + pub fn send(&self, socket: UdpSockRef<'_>, transmit: &Transmit<'_>) -> io::Result<()> { + match send(socket, transmit) { + Ok(()) => Ok(()), + Err(e) if e.kind() == io::ErrorKind::WouldBlock => Err(e), + Err(e) => { + log_sendmsg_error(&self.last_send_error, e, transmit); + + Ok(()) + } + } + } + + pub fn try_send(&self, socket: UdpSockRef<'_>, transmit: &Transmit<'_>) -> io::Result<()> { + send(socket, transmit) + } + + pub fn recv( + &self, + socket: UdpSockRef<'_>, + bufs: &mut [IoSliceMut<'_>], + meta: &mut [RecvMeta], + ) -> io::Result { + let bufs = unsafe { + &mut *(bufs as *mut [IoSliceMut<'_>] as *mut [socket2::MaybeUninitSlice<'_>]) + }; + let (len, _flags, addr) = socket.0.recv_from_vectored(bufs)?; + meta[0] = RecvMeta { + len, + stride: len, + addr: addr.as_socket().unwrap(), + ecn: None, + dst_ip: None, + }; + Ok(1) + } + + #[inline] + pub fn max_gso_segments(&self) -> usize { + 1 + } + + #[inline] + pub fn gro_segments(&self) -> usize { + 1 + } + + #[inline] + pub fn set_send_buffer_size(&self, socket: UdpSockRef<'_>, bytes: usize) -> io::Result<()> { + socket.0.set_send_buffer_size(bytes) + } + + #[inline] + pub fn set_recv_buffer_size(&self, socket: UdpSockRef<'_>, bytes: usize) -> io::Result<()> { + socket.0.set_recv_buffer_size(bytes) + } + + #[inline] + pub fn send_buffer_size(&self, socket: UdpSockRef<'_>) -> io::Result { + socket.0.send_buffer_size() + } + + #[inline] + pub fn recv_buffer_size(&self, socket: UdpSockRef<'_>) -> io::Result { + socket.0.recv_buffer_size() + } + + #[inline] + pub fn may_fragment(&self) -> bool { + true + } +} + +fn send(socket: UdpSockRef<'_>, transmit: &Transmit<'_>) -> io::Result<()> { + socket.0.send_to( + transmit.contents, + &socket2::SockAddr::from(transmit.destination), + )?; + Ok(()) +} + +pub(crate) const BATCH_SIZE: usize = 1; diff --git a/shadow/quinn-udp-patch/src/lib.rs b/shadow/quinn-udp-patch/src/lib.rs new file mode 100644 index 00000000..4292d284 --- /dev/null +++ b/shadow/quinn-udp-patch/src/lib.rs @@ -0,0 +1,150 @@ +//! Minimal `quinn-udp` replacement used only for Shadow-simulator builds. +//! +//! Upstream `quinn-udp` reaches for GSO/GRO batch syscalls (`sendmmsg`, +//! `recvmmsg`, segmentation offload) on Linux, which the [Shadow] network +//! simulator does not emulate. This crate keeps the upstream public API but +//! routes every send/receive through plain `send_to`/`recv_from` (batch size +//! 1), so QUIC works under Shadow at the cost of per-packet syscalls. +//! +//! It is **not** part of normal builds: a Cargo `[patch.crates-io]` table +//! cannot be feature-gated, so it is injected into the workspace manifest only +//! for Shadow builds (`shadow/build.sh` / `make shadow-build`), kept in sync +//! with the `shadow-integration` Cargo feature. +//! +//! Used for integration with the [lean-shadow-fuzzer], which drives ethlambda +//! under Shadow to fuzz the lean consensus network. +//! +//! [Shadow]: https://shadow.github.io/ +//! [lean-shadow-fuzzer]: https://github.com/kamilsa/lean-shadow-fuzzer + +#![warn(unreachable_pub)] +#![warn(clippy::use_self)] + +use std::net::{IpAddr, Ipv6Addr, SocketAddr}; +#[cfg(unix)] +use std::os::unix::io::AsFd; +use std::{ + sync::Mutex, + time::{Duration, Instant}, +}; + +#[path = "fallback.rs"] +mod imp; + +#[allow(unused_imports, unused_macros)] +mod log { + #[cfg(all(feature = "direct-log", not(feature = "tracing")))] + pub(crate) use log::{debug, error, info, trace, warn}; + + #[cfg(feature = "tracing")] + pub(crate) use tracing::{debug, error, info, trace, warn}; + + #[cfg(not(any(feature = "direct-log", feature = "tracing")))] + mod no_op { + macro_rules! trace ( ($($tt:tt)*) => {{}} ); + macro_rules! debug ( ($($tt:tt)*) => {{}} ); + macro_rules! info ( ($($tt:tt)*) => {{}} ); + macro_rules! log_warn ( ($($tt:tt)*) => {{}} ); + macro_rules! error ( ($($tt:tt)*) => {{}} ); + + pub(crate) use {debug, error, info, log_warn as warn, trace}; + } + + #[cfg(not(any(feature = "direct-log", feature = "tracing")))] + pub(crate) use no_op::*; +} + +pub use imp::UdpSocketState; + +pub const BATCH_SIZE: usize = imp::BATCH_SIZE; + +#[derive(Debug, Copy, Clone)] +pub struct RecvMeta { + pub addr: SocketAddr, + pub len: usize, + pub stride: usize, + pub ecn: Option, + pub dst_ip: Option, +} + +impl Default for RecvMeta { + fn default() -> Self { + Self { + addr: SocketAddr::new(Ipv6Addr::UNSPECIFIED.into(), 0), + len: 0, + stride: 0, + ecn: None, + dst_ip: None, + } + } +} + +#[derive(Debug, Clone)] +pub struct Transmit<'a> { + pub destination: SocketAddr, + pub ecn: Option, + pub contents: &'a [u8], + pub segment_size: Option, + pub src_ip: Option, +} + +const IO_ERROR_LOG_INTERVAL: Duration = std::time::Duration::from_secs(60); + +#[cfg(all(any(feature = "tracing", feature = "direct-log")))] +fn log_sendmsg_error( + last_send_error: &Mutex, + err: impl core::fmt::Debug, + transmit: &Transmit, +) { + let now = Instant::now(); + let last_send_error = &mut *last_send_error.lock().expect("poisend lock"); + if now.saturating_duration_since(*last_send_error) > IO_ERROR_LOG_INTERVAL { + *last_send_error = now; + log::warn!( + "sendmsg error: {:?}, Transmit: {{ destination: {:?}, src_ip: {:?}, ecn: {:?}, len: {:?}, segment_size: {:?} }}", + err, + transmit.destination, + transmit.src_ip, + transmit.ecn, + transmit.contents.len(), + transmit.segment_size + ); + } +} + +#[cfg(not(any(feature = "tracing", feature = "direct-log")))] +fn log_sendmsg_error(_: &Mutex, _: impl core::fmt::Debug, _: &Transmit) {} + +pub struct UdpSockRef<'a>(socket2::SockRef<'a>); + +#[cfg(unix)] +impl<'s, S> From<&'s S> for UdpSockRef<'s> +where + S: AsFd, +{ + fn from(socket: &'s S) -> Self { + Self(socket.into()) + } +} + +#[repr(u8)] +#[derive(Debug, Copy, Clone, Eq, PartialEq)] +pub enum EcnCodepoint { + Ect0 = 0b10, + Ect1 = 0b01, + Ce = 0b11, +} + +impl EcnCodepoint { + pub fn from_bits(x: u8) -> Option { + use EcnCodepoint::*; + Some(match x & 0b11 { + 0b10 => Ect0, + 0b01 => Ect1, + 0b11 => Ce, + _ => { + return None; + } + }) + } +} From e9144621bbc2508269027f3941f742c471f87a18 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1s=20Gr=C3=BCner?= <47506558+MegaRedHand@users.noreply.github.com> Date: Mon, 8 Jun 2026 15:25:48 -0300 Subject: [PATCH 2/2] docs: improve why table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Tomás Grüner <47506558+MegaRedHand@users.noreply.github.com> --- shadow/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/shadow/README.md b/shadow/README.md index efa0b7f9..b8af5d91 100644 --- a/shadow/README.md +++ b/shadow/README.md @@ -12,9 +12,9 @@ things in a stock ethlambda build need to change for Shadow: | Area | Stock build | Under Shadow | Why | |------|-------------|--------------|-----| -| Allocator | jemalloc with `unprefixed_malloc` (interposes the global C `malloc`) | system allocator (drop jemalloc) | **Correctness.** Self-deadlocks at startup ([shadow#3763]): Shadow's shim `fopen`s `/proc/self/maps` on its first intercepted syscall, calling `malloc`, which re-enters jemalloc mid-init while it holds its non-recursive init lock. | -| QUIC UDP I/O | `quinn-udp` uses GSO/GRO batch syscalls (`sendmmsg`, segmentation offload) | fall back to `send_to`/`recv_from` | **Correctness.** Shadow's UDP emulation doesn't support those batch syscalls. | -| Tokio runtime | multi-threaded | `current_thread` | **Optimization only.** Shadow single-steps execution, so worker threads add only scheduling noise, never parallelism. | +| Allocator | jemalloc with `unprefixed_malloc` (interposes the global C `malloc`) | system allocator (drop jemalloc) | Self-deadlocks at startup ([shadow#3763]) | +| QUIC UDP I/O | `quinn-udp` uses GSO/GRO batch syscalls (`sendmmsg`, segmentation offload) | fall back to `send_to`/`recv_from` | Shadow's UDP emulation doesn't support those batch syscalls. | +| Tokio runtime | multi-threaded | `current_thread` | Shadow single-steps execution, so worker threads add only scheduling noise. | The allocator change (dropping jemalloc) and the runtime flavor are gated behind the `shadow-integration` Cargo feature (jemalloc is dropped via