LCORE-2343: implement behave step definitions for unified-mode feature files

docs/testing/e2e_testing.md

@@ -15,8 +15,9 @@ This guide describes how to run, extend, and understand the Lightspeed Core Stac
 7. [Configuration Files](#configuration-files)
 8. [Feature Files and Steps](#feature-files-and-steps)
 9. [Gherkin Keywords in Feature Files](#gherkin-keywords-in-feature-files)
-10. [Writing New Scenarios](#writing-new-scenarios)
-11. [Troubleshooting](#troubleshooting)
+10. [Choosing the Test Layer: E2E or Integration?](#choosing-the-test-layer-e2e-or-integration)
+11. [Writing New Scenarios](#writing-new-scenarios)
+12. [Troubleshooting](#troubleshooting)
 
 ---
 
@@ -336,9 +337,50 @@ Here, **Given** sets state, **When** performs the HTTP call, **Then** and **And*
 
 ---
 
+## Choosing the Test Layer: E2E or Integration?
+
+Before writing a scenario, decide whether it belongs here at all. The suite has
+three layers, and the boundary between the top two is strict:
+
+| Layer | Location | Talks to | May touch `src/`? |
+|---|---|---|---|
+| Unit | `tests/unit/` | one function or class, everything else mocked | yes |
+| Integration | `tests/integration/` (pytest) | real configuration loading, real database, real pipelines in-process; external services (OGX, LLM providers) mocked; repo CLIs as subprocesses | yes |
+| E2E | `tests/e2e/` (behave) | a deployed stack, through its public surfaces only: the HTTP API, container lifecycle and logs, configuration files the harness applies | **never** |
+
+The rule for e2e: **a step definition must not import from, invoke, or shell out
+to anything under `src/`.** The moment it does, the scenario stops proving what a
+deployed stack does and starts proving what a checked-out source tree does — that
+is an integration test, and it belongs in `tests/integration/` as pytest, where it
+runs in seconds without Docker.
+
+A quick test: *could this scenario run unchanged against a container image, with
+no source checkout on the machine?* If yes, it is e2e. If it needs
+`src/lightspeed_stack.py`, `src/ogx_configuration.py`, a Python import from the
+service, or a subprocess of a repo entrypoint, it is integration.
+
+Typical consequences:
+
+- Configuration **validation**, **migration** (`--migrate-config`) and run.yaml
+  **synthesis** are integration concerns: they exercise CLIs and the config
+  pipeline, not a running service. See `tests/integration/test_unified_synthesis.py`
+  and `tests/integration/test_unified_mode_cli.py`.
+- **Boot** scenarios (apply a config, restart, hit `readiness` and `query`) and
+  **log-evidence** scenarios (`docker logs <container>`) are e2e: they observe the
+  deployed stack from outside.
+- If a scenario needs a generated artifact as its starting point (for example a
+  migrated configuration), commit the artifact as a fixture and add an
+  integration test that guards it against drift, rather than generating it inside
+  the e2e step.
+
+The integration side of this boundary is described in
+[tests/integration/README.md](../../tests/integration/README.md#what-to-test).
+
+---
+
 ## Writing New Scenarios
 
-1. **Choose or add a feature file** under `tests/e2e/features/` and use existing steps where possible. If you add a new file, **add it to `tests/e2e/test_list.txt`** so the suite runs it. 
+1. **Confirm the scenario is e2e at all** — see [Choosing the Test Layer](#choosing-the-test-layer-e2e-or-integration). Then **choose or add a feature file** under `tests/e2e/features/` and use existing steps where possible. If you add a new file, **add it to `tests/e2e/test_list.txt`** so the suite runs it. 
 2. **Use tags** for mode-dependent or config-dependent behavior (`@skip-in-library-mode`, `@Authorized`, etc.). **Adding a tag that switches configuration** (e.g. a new feature-level or scenario-level config) usually means you must also add or change a **Lightspeed Stack config** file under `configuration/server-mode/` or `library-mode/` and wire the tag in `environment.py` (e.g. in `before_feature` / `after_feature` or `before_scenario` / `after_scenario`) so the config is applied and the container restarted when the tag is active.
 3. **Use placeholders** `{MODEL}` and `{PROVIDER}` in request bodies so the same scenario works with different backends.
 4. **Add step definitions** in the appropriate `features/steps/*.py` if you need new steps; reuse `context` for host, port, auth, and responses.

docs/testing/testing.md

@@ -132,6 +132,7 @@ As specified in Definition of Done, new changes need to be covered by tests.
 Integration tests are based on the [Pytest framework](https://docs.pytest.org/en/) and code coverage is measured by the plugin [pytest-cov](https://github.com/pytest-dev/pytest-cov). For mocking and patching, the [unittest framework](https://docs.python.org/3/library/unittest.html) is used.
 
 * Defined in [tests/integration](https://github.com/lightspeed-core/lightspeed-stack/tree/main/tests/integration)
+* **Integration or e2e?** Integration tests may touch `src/` (in-process pipelines, repo CLIs as subprocesses); e2e tests never do. See [Choosing the Test Layer](e2e_testing.md#choosing-the-test-layer-e2e-or-integration).
 
 
 

src/ogx_configuration.py

@@ -30,7 +30,7 @@
 from pydantic import SecretStr
 
 import constants
-from log import get_logger
+from log import get_logger, setup_logging
 
 logger = get_logger(__name__)
 
@@ -1550,6 +1550,10 @@ def main() -> None:
     run.yaml needs to exist; otherwise the legacy path enriches the
     ``--input`` run.yaml in place. Server-mode container entrypoints rely on
     this dispatch to serve both modes with a single invocation.
+
+    Configures logging first so the INFO lines this module emits reach the
+    container log: run as a bare script there is no handler on the root
+    logger, and ``logging.lastResort`` would drop everything below WARNING.
     """
     parser = ArgumentParser(
         description="Generate the OGX run configuration from a "
@@ -1577,6 +1581,15 @@ def main() -> None:
     )
     args = parser.parse_args()
 
+    # Configure logging before doing any work. This module runs as a bare
+    # script from the container entrypoint (scripts/ogx-entrypoint.sh), so
+    # nothing has installed a handler on the root logger; Python's lastResort
+    # then emits WARNING and above only, and every INFO line this module
+    # writes -- including which config shape was detected and where the
+    # synthesized run.yaml was written -- is silently dropped. AsyncOgxClient
+    # already does this for the in-process path, for the same reason.
+    setup_logging()
+
     with open(args.config, encoding="utf-8") as f:
         config = yaml.safe_load(f)
 

tests/configuration/unified-mode/lightspeed-stack-invalid-config-and-legacy.yaml

@@ -0,0 +1,25 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  config:
+    profile: tests/configuration/run.yaml
+  # INVALID: config block plus the legacy path (mutual exclusion, R3)
+  library_client_config_path: tests/configuration/run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini

tests/configuration/unified-mode/lightspeed-stack-invalid-providers-and-legacy.yaml

@@ -0,0 +1,31 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  # INVALID: synthesis input plus the legacy path (mutual exclusion, R3)
+  library_client_config_path: tests/configuration/run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini
+  # Unified synthesis input (Decision S5): the high-level provider entry
+  # replaces the default baseline's openai provider by id at synthesis time.
+  providers:
+    - type: openai
+      id: openai
+      api_key_env: OPENAI_API_KEY
+      allowed_models:
+        - ${env.E2E_OPENAI_MODEL:=gpt-4o-mini}

tests/configuration/unified-mode/lightspeed-stack-invalid-version-legacy-unified-body.yaml

@@ -0,0 +1,31 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini
+  # Unified synthesis input (Decision S5): the high-level provider entry
+  # replaces the default baseline's openai provider by id at synthesis time.
+  providers:
+    - type: openai
+      id: openai
+      api_key_env: OPENAI_API_KEY
+      allowed_models:
+        - ${env.E2E_OPENAI_MODEL:=gpt-4o-mini}
+# INVALID: explicit legacy marker on a unified-shaped body (R11, LCORE-2872)
+config_format_version: legacy

tests/configuration/unified-mode/lightspeed-stack-legacy-for-migration.yaml

@@ -0,0 +1,23 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  # Legacy two-file shape: external run.yaml, no synthesis input
+  library_client_config_path: tests/e2e/configs/run-ci.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini

tests/e2e/README.md

@@ -2,8 +2,10 @@
 
 End-to-end tests for the Lightspeed Core Stack REST API (Behave, Gherkin).
 
-**Full guide:** [docs/e2e_testing.md](../../docs/e2e_testing.md) — how to run, environment variables, deployment modes, tags and hooks, Gherkin keywords, configuration, and troubleshooting.
+**Full guide:** [docs/testing/e2e_testing.md](../../docs/testing/e2e_testing.md) — how to run, environment variables, deployment modes, tags and hooks, Gherkin keywords, configuration, and troubleshooting.
 
 * Tests: `tests/e2e/features/*.feature`
 * Step definitions: `tests/e2e/features/steps/`
 * Feature list (run order): `test_list.txt`
+
+**Not sure a scenario is e2e?** Steps must never touch `src/`; validation, migration and synthesis live in `tests/integration/`. See [Choosing the Test Layer](../../docs/testing/e2e_testing.md#choosing-the-test-layer-e2e-or-integration).

tests/e2e/configuration/library-mode/lightspeed-stack-legacy.yaml

@@ -0,0 +1,47 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Legacy two-file shape (R2 deprecation window): external run.yaml consumed
+  # via library_client_config_path; no unified synthesis input. Kept as a
+  # dedicated fixture because the standard library-mode baseline migrated to
+  # unified mode in LCORE-2342, which silently removed legacy boot coverage.
+  use_as_library_client: true
+  library_client_config_path: run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini
+rag:
+  byok:
+    stores:
+      - rag_id: e2e-test-docs
+        backend: faiss
+        embedding_model: sentence-transformers/all-mpnet-base-v2
+        embedding_dimension: 768
+        vector_db_id: ${env.FAISS_VECTOR_STORE_ID}
+        db_path: ${env.KV_RAG_PATH:=~/.llama/storage/rag/kv_store.db}
+        score_multiplier: 1.0
+  retrieval:
+    tool:
+      sources:
+        - e2e-test-docs
+
+shields:
+  - name: pii-redaction
+    provider_id: redaction
+    config:
+      rules:
+        - pattern: '\d+'
+          replacement: '[NUM]'

tests/e2e/configuration/unified-mode/README.md

@@ -0,0 +1,25 @@
+# Unified-mode e2e configuration fixtures
+
+Fixtures for the `unified-mode-*.feature` files (LCORE-2341/LCORE-2343).
+Same layout as the parent directory: `library-mode/` and `server-mode/`
+variants differing only in the `llama_stack` block; the harness resolves
+`<dir>/<mode>/<file>` via the standard `configure_service` logic.
+
+All profile-based fixtures reference `run.yaml` — the repo-root copy the CI
+harness materializes from `tests/e2e/configs/run-<env>.yaml` — so they stay
+provider-agnostic across the providers matrix.
+
+Only bootable fixtures live here. The validation-only and synthesis-only
+inputs (invalid configs, `native_override` shapes) belong to the integration
+layer — `tests/configuration/unified-mode/` and
+`tests/integration/test_unified_synthesis.py` — because e2e steps never run
+`src/` CLIs (see `docs/testing/e2e_testing.md`, "Choosing the Test Layer").
+
+| Fixture | Purpose |
+|---|---|
+| `lightspeed-stack-unified-providers.yaml` | Minimal unified config driven only by top-level `inference.providers` (default baseline, R1/S5). openai-specific — used by `@openai-only` scenarios. |
+| `lightspeed-stack-unified-config-only.yaml` | Unified config driven only by `llama_stack.config` (`profile: run.yaml`, R1). |
+| `lightspeed-stack-unified-relative-profile.yaml` | Same shape as config-only; exists to pin R8 (relative `profile:` resolves against the config file's directory) as a distinct intent. |
+| `lightspeed-stack-unified-absolute-profile.yaml` | `profile:` as a container-absolute path (differs per mode subdir). |
+| `lightspeed-stack-legacy-for-migration.yaml` | Legacy half of the migration fixture pair; paired with `tests/e2e/configs/run-ci.yaml`. Deliberately free of enrichment sections so migrate→synthesize round-trips losslessly (see LCORE-3370). Input to the drift guard below; never booted. |
+| `lightspeed-stack-unified-migrated.yaml` | **Committed** output of `--migrate-config` for the pair above. Booted by `unified-mode-migration.feature` (`@openai-only`: it inlines the openai run-ci.yaml). `tests/integration/test_unified_mode_cli.py::test_committed_migrated_fixture_matches_cli_output` fails when the CLI output drifts; its docstring has the regeneration command. |

tests/e2e/configuration/unified-mode/library-mode/lightspeed-stack-legacy-for-migration.yaml

@@ -0,0 +1,23 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  # Legacy two-file shape: external run.yaml, no synthesis input
+  library_client_config_path: run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini

tests/e2e/configuration/unified-mode/library-mode/lightspeed-stack-unified-absolute-profile.yaml

@@ -0,0 +1,24 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  config:
+    # Absolute path as mounted in the library-mode container
+    profile: /app-root/run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini

tests/e2e/configuration/unified-mode/library-mode/lightspeed-stack-unified-config-only.yaml

@@ -0,0 +1,24 @@
+name: Lightspeed Core Service (LCS)
+service:
+  host: 0.0.0.0
+  port: 8080
+  auth_enabled: false
+  workers: 1
+  color_log: true
+  access_log: true
+llama_stack:
+  # Library mode - embeds the stack in-process
+  use_as_library_client: true
+  config:
+    # Synthesis baseline: the CI-materialized run.yaml (provider-agnostic)
+    profile: run.yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+authentication:
+  module: "noop"
+inference:
+  default_provider: openai
+  default_model: gpt-4o-mini