feat: Deferred skills + skill search: load only a few top skills at start, update the system prompt to look up skills (scale t

CHANGELOG.md

@@ -42,8 +42,60 @@ Known evidence gaps:
 
 ## [Unreleased]
 
+### Added
+
+- Skills now scale to large catalogs. When the full skill catalog does not fit
+  the 2% skills context budget, the developer message renders a small starter
+  list instead of every skill, and the model reaches the rest through a new
+  `skills.list` catalog-search tool (available in the app-server and MCP-server
+  extension registries). The starter list is dealt round-robin across skill
+  scopes, so bundled system skills cannot starve a workspace's own skills.
+- `skills.list` supports paging: `limit` (default 5, max 50) and an unbounded
+  `offset`, with a `next_offset` cursor in the response, so every ranked match
+  is reachable without rewording the query. The walk is bounded by the catalog
+  and nothing else - there is no offset ceiling, `next_offset` always strictly
+  advances, and any cursor the tool emits is a cursor it accepts. Responses
+  carry `total_matches` and `has_more` so exhaustion is distinguishable from a
+  partial page. Ranking is lexical token overlap with light plural/verb-form
+  stemming, so `charts` finds a `chart` skill; it has no synonym or embedding
+  layer.
+- `skills.list` can enumerate. `query` is optional: omit it (or pass an empty
+  string) to page through the whole catalog alphabetically. Lexical ranking can
+  only surface skills whose metadata shares a token with the request, so without
+  enumeration a skill whose wording the model cannot guess was unreachable in
+  principle, not merely hard to find.
+- New `codex.thread.skills.deferred_total` metric reports how many skills were
+  held back from the starter list. `codex.thread.skills.truncated` now also
+  flags deferral, so a large catalog can no longer report a big drop in kept
+  skills alongside `truncated = 0`.
+
 ### Fixed
 
+- The starter-list cap only applies when it is actually needed: a catalog that
+  can be listed at all inside the skills context budget still renders in full
+  (with descriptions shortened as required) rather than hiding skills, and an
+  embedder that installs no `skills.list` tool always gets the complete list
+  rather than a truncated one it cannot search past.
+- Deferring skills is never silent. Whenever the starter cap holds skills back,
+  the session emits a warning naming how many skills are shown, how many exist,
+  and that the rest stay searchable.
+- The `## Skills` intro no longer claims the list is incomplete when it is
+  complete, and no longer points at `skills.list` on threads where that tool is
+  not installed.
+- The per-turn ranked skills fragment carries the usage rules itself when no
+  `## Skills` developer block was emitted. Core builds that block from the host
+  skill outcome, so a catalog served entirely by remote or executor providers
+  used to leave the model with bare skill lines, no usage rules, and a reference
+  to a section that did not exist.
+- The average description-truncation telemetry is no longer diluted by the full
+  catalog size, which rounded real per-skill truncation down to zero on large
+  catalogs and disarmed the truncation warning.
+- The per-turn ranked skills fragment no longer repeats the `How to use skills`
+  preamble already present in the developer-message skills block, saving roughly
+  2.5k characters on every turn that matches a skill.
+- Building the host skill catalog is no longer quadratic. It ran on every turn
+  and de-duplicated with a linear scan per insert over long common-prefix
+  filesystem paths.
 - Auth-profile auto-switch now resumes the interrupted turn on the new profile.
   When a turn fails on a rate-limited profile and no usage reset is available,
   the agent switches to a healthier profile and automatically re-runs the failed

codex-rs/Cargo.toml

@@ -223,6 +223,7 @@ codex-secrets = { path = "secrets" }
 codex-shell-command = { path = "shell-command" }
 codex-shell-escalation = { path = "shell-escalation" }
 codex-skills = { path = "skills" }
+codex-skills-extension = { path = "ext/skills" }
 codex-state = { path = "state" }
 codex-stdio-to-uds = { path = "stdio-to-uds" }
 codex-terminal-detection = { path = "terminal-detection" }

codex-rs/app-server/Cargo.toml

@@ -51,6 +51,7 @@ codex-hooks = { workspace = true }
 codex-otel = { workspace = true }
 codex-plugin = { workspace = true }
 codex-shell-command = { workspace = true }
+codex-skills-extension = { workspace = true }
 codex-utils-cli = { workspace = true }
 codex-utils-pty = { workspace = true }
 codex-backend-client = { workspace = true }

codex-rs/app-server/src/extensions.rs

@@ -86,6 +86,7 @@ where
     }
     codex_guardian::install(&mut builder, guardian_agent_spawner);
     codex_memories_extension::install(&mut builder, codex_otel::global());
+    codex_skills_extension::install(&mut builder);
     codex_web_search_extension::install(&mut builder, auth_manager.clone());
     codex_image_generation_extension::install(&mut builder, auth_manager);
     codex_workflows_extension::install(&mut builder, workflow_state_db, |config: &Config| {

codex-rs/app-server/tests/suite/v2/turn_start.rs

@@ -1017,14 +1017,25 @@ async fn turn_start_emits_thread_scoped_warning_notification_for_trimmed_skills(
         "Exceeded skills context budget of 2%. All skill descriptions were removed and ";
     let warning_suffix = " additional skills were not included in the model-visible skills list.";
     assert!(
-        warning.message.starts_with(warning_prefix) && warning.message.ends_with(warning_suffix),
+        warning.message.starts_with(warning_prefix),
         "unexpected warning message: {:?}",
         warning.message
     );
-    let omitted_skill_count = warning
+    // A budget this small also trips the starter cap, so the omission sentence
+    // is followed by the deferral notice. Both have to be there: the user needs
+    // to know skills are missing *and* that `skills.list` can still find them.
+    let (omission, deferral) = warning
         .message
+        .split_once(&format!("{warning_suffix} "))
+        .unwrap_or_else(|| panic!("unexpected warning message: {:?}", warning.message));
+    assert!(
+        deferral.starts_with("Showing ")
+            && deferral.ends_with("stay searchable through `skills.list`."),
+        "expected a deferral notice: {:?}",
+        warning.message
+    );
+    let omitted_skill_count = omission
         .trim_start_matches(warning_prefix)
-        .trim_end_matches(warning_suffix)
         .parse::<usize>()
         .expect("warning omitted skill count should be numeric");
     assert!(

codex-rs/core-skills/src/lib.rs

@@ -22,13 +22,36 @@ pub use model::SkillMetadata;
 pub use model::SkillPolicy;
 pub use model::filter_skill_load_outcome_for_product;
 pub use render::AvailableSkills;
-pub use render::SKILLS_HOW_TO_USE_WITH_ABSOLUTE_PATHS;
-pub use render::SKILLS_HOW_TO_USE_WITH_ALIASES;
-pub use render::SKILLS_INTRO_WITH_ABSOLUTE_PATHS;
-pub use render::SKILLS_INTRO_WITH_ALIASES;
+pub use render::MAX_STARTER_SKILLS;
+pub use render::SKILLS_DISCOVERY_CATALOG_SEARCH;
+pub use render::SKILLS_DISCOVERY_COMPLETE_LIST;
+pub use render::SKILLS_DISCOVERY_LEAD;
+pub use render::SKILLS_DISCOVERY_PARTIAL_LIST;
+pub use render::SKILLS_DISCOVERY_PATHS_WITH_ABSOLUTE_PATHS;
+pub use render::SKILLS_DISCOVERY_PATHS_WITH_ALIASES;
+pub use render::SKILLS_HOW_TO_USE_TAIL_WITH_ABSOLUTE_PATHS;
+pub use render::SKILLS_HOW_TO_USE_TAIL_WITH_ALIASES;
+pub use render::SKILLS_INTRO_COMPLETE_LIST;
+pub use render::SKILLS_INTRO_LEAD;
+pub use render::SKILLS_INTRO_PARTIAL_LIST;
+pub use render::SKILLS_INTRO_TRAILER_WITH_ABSOLUTE_PATHS;
+pub use render::SKILLS_INTRO_TRAILER_WITH_ALIASES;
+pub use render::SKILLS_LIST_TOOL_NAME;
+pub use render::SKILLS_MISSING_WITH_CATALOG_SEARCH;
+pub use render::SKILLS_MISSING_WITHOUT_CATALOG_SEARCH;
+pub use render::SKILLS_TOOL_NAMESPACE;
+pub use render::SKILLS_TRIGGER_RULES;
+pub use render::SkillCatalogSearch;
 pub use render::SkillMetadataBudget;
 pub use render::SkillRenderReport;
+pub use render::SkillsListCoverage;
+pub use render::SkillsPreamble;
+pub use render::TASK_RELEVANT_SKILLS_HEADING;
+pub use render::TASK_RELEVANT_SKILLS_INTRO;
+pub use render::TASK_RELEVANT_SKILLS_STANDALONE_INTRO;
 pub use render::build_available_skills;
+pub use render::build_available_skills_with_starter_cap;
 pub use render::default_skill_metadata_budget;
 pub use render::render_available_skills_body;
+pub use render::render_task_relevant_skills_body;
 pub use skill_instructions::SkillInstructions;