Reliable generative programs with Mellea

A production-shaped learning project for building reliable generative programs. It uses Mellea to analyze customer-support tickets, apply deterministic business policy, and draft replies that are validated, repaired, or explicitly rejected.

The companion tutorial is available from the TechnologyDig Academy.

What makes this a generative program?

Most of the application is ordinary Python. Two operations ask a language model to compute a result, so those operations are nondeterministic. Mellea makes their contracts visible:

trusted ticket
    │
    ├── deterministic fact extraction ───────────────┐
    │                                                │
    └── @generative typed interpretation             │
                    │                                │
                    ▼                                │
             deterministic policy                    │
                    │                                │
                    ▼                                │
          instruct → validate → repair ◄─────────────┘
                    │
             pass ──┴── exhausted
              │            │
        validated reply    explicit failure

The model interprets ambiguous language. Python owns identifiers, financial authorization, routing policy, and the final decision about whether a result is safe to return.

What you will learn

Concept	Implementation	Lesson
Typed generative function	analyze_ticket	A signature, docstring, and return type define a reusable model-backed operation.
Structured output	TicketAnalysis	A valid schema guarantees shape, not factual correctness.
Deterministic policy	decide_route	Models interpret; auditable code decides.
Trusted data lineage	extract_order_ids	Validate claims against original input, not another generated value.
Requirements	draft_reply	Constraints are executable checks with actionable failure reasons.
Instruct–Validate–Repair	RepairTemplateStrategy	Failed checks inform the next bounded generation attempt.
Explicit failure	ReplyValidationError	A selected failed sample is diagnostic data, not approval.
Test seams	tests/	Unit-test deterministic code and evaluate model behavior separately.

Prerequisites

• Python 3.11 or 3.12
• uv
• Ollama for the default local backend

Install the default model and dependencies:

ollama pull granite4.1:3b
uv sync --extra dev

No API key is required for the default local path.

Start with the narrated workshop

The workshop pauses at every deterministic/generative boundary, prints the Python type returned by the model-backed function, and exposes every validation attempt:

uv run mellea-workshop examples/ticket_refund.json

To run it without pauses:

uv run mellea-workshop examples/ticket_refund.json --no-pause

Then work through LEARNING_PATH.md. Its labs follow a prediction, edit, observation loop rather than asking you to read finished code passively.

Run the application

The application-style CLI prints the final structured result:

uv run support-copilot examples/ticket_refund.json

Run only the typed analysis step:

uv run support-copilot examples/ticket_account.json --analysis-only

Switch to another Mellea backend and model:

uv run support-copilot examples/ticket_refund.json \
  --backend openai \
  --model gpt-4o-mini

The relevant provider credential, such as OPENAI_API_KEY, must already be configured. This project never stores credentials.

Follow the program through the files

1. models.py defines trusted input, generative output, policy, and workflow contracts.
2. facts.py extracts identifiers whose syntax does not require a model.
3. mellea_program.py contains the two nondeterministic boundaries.
4. policy.py contains auditable business decisions.
5. validators.py returns pass/fail decisions and repair reasons.
6. workflow.py composes everything using ordinary Python.
7. learning.py makes normally internal execution visible.

Reliability contracts

This project distinguishes contracts that are often collapsed into one prompt:

Contract	Question
Input	What trusted data enters the operation?
Structural	What Python type must come back?
Behavioral	Which requirements must the result pass?
Policy	Which decisions must remain deterministic?
Provenance	What source is authoritative for a factual claim?
Failure	What happens after the generation budget is exhausted?
Evaluation	How will semantic quality be measured over time?

Mellea directly supports typed operations, requirements, sampling, context, and result inspection. The application still owns policy, provenance, and the meaning of failure.

Test without pretending the model is deterministic

uv run pytest -q
uv run ruff check .

The unit suite does not call Ollama or a paid backend. It verifies:

• routing and escalation policy;
• deterministic identifier extraction;
• reply validators and repair feedback;
• requirement wiring;
• budget-exhaustion handling;
• workflow composition through an injected fake analyzer.

Model quality needs a separate labelled evaluation set. Schema validity, requirement pass rate, and semantic accuracy are different measurements.

Run the included eight-case live evaluation once:

uv run mellea-eval --runs 1

Repeat every case to expose stochastic variation:

uv run mellea-eval --runs 3

The evaluation calls the configured model and reports category, urgency, and safety-risk accuracy. Its small fixture teaches the mechanism; it is not a production benchmark.

Failure is part of the interface

Mellea's SamplingResult contains a selected result even when no attempt satisfies every requirement. The selected candidate is useful for inspection, but SamplingResult.success decides whether the application may return it. This project raises ReplyValidationError on exhaustion and quarantines the candidate in workshop mode.

The project also validates generated order IDs against the original ticket. An earlier version validated one model output against another, causing a correct source ID to fail when the first model call missed it. A deterministic validator can still be wrong when its reference data is wrong.

Production boundaries

This is a learning system, not an autonomous support agent:

• Human review is required for consequential cases.
• The model never authorizes money or account actions.
• Retry budgets bound cost and latency but do not guarantee success.
• Structured output guarantees shape but not truth.
• Production deployments should record model/version, validation outcomes, attempts, latency, and human edits while redacting personal data.

Primary references

• Mellea quick start
• Generative programming
• Generative functions
• Requirements system
• Structured output
• Backends and configuration

License

Copyright 2026 TechnologyDig. Licensed under the Apache License 2.0.