diff --git a/skills/cuopt-multi-objective-exploration/SKILL.md b/skills/cuopt-multi-objective-exploration/SKILL.md index 61daf25574..099eb444e4 100644 --- a/skills/cuopt-multi-objective-exploration/SKILL.md +++ b/skills/cuopt-multi-objective-exploration/SKILL.md @@ -1,7 +1,7 @@ --- name: cuopt-multi-objective-exploration version: "26.08.00" -description: Trace and interpret the Pareto frontier across competing objectives using repeated single-objective cuOpt solves (weighted-sum and ε-constraint). +description: Trace, complete, and interpret the Pareto frontier across competing objectives using repeated single-objective cuOpt solves (weighted-sum and ε-constraint). license: Apache-2.0 origin: cuopt-skill-evolution metadata: @@ -123,11 +123,42 @@ Practical notes: - **Spend the budget where the slope changes (LP/QP).** Because the ε-constraint dual is the frontier's local slope, compare it across solved points: where it barely changes, the curve is nearly straight — interpolate rather than add solves; where it jumps by more than the solve tolerance, the frontier bends between those points — refine there (smaller differences are solver noise, not curvature). This concentrates solves where the curve actually bends instead of spreading them over a uniform grid. On MILP, judge where to refine from the gaps between primal objective values instead. - **Verify, don't assume.** When you claim one method beats another, measure it — e.g. count the efficient points ε-constraint recovered that weighted-sum missed — rather than asserting it; and flag any solve returning feasible-but-not-`Optimal` so a non-certified point is never read as exact. -## Step 5 — interpret the frontier +## Step 5 — complete the frontier: measure and fill what the sweep missed + +A weighted-sum sweep returns only **supported** points (Step 3's convex-hull limitation); on MILP frontiers, non-supported points — the ones no weighted-sum weighting returns — often make up much of the non-dominated set. A coarse ε-constraint grid leaves gaps the same way: any finite sweep can miss regions. Before presenting a swept frontier, measure the likely miss and decide whether to fill. + +### Measure the miss + +Sort the swept points by one objective. For each adjacent pair, form the rectangle (in general, the box) between them in objective space; flag any box much larger than the median adjacent box (3× is a reasonable bar) or covering a large share of the frontier's spanned area — a sweep that returned only a handful of points is all gaps, so no box stands out from the median. Large boxes have two causes — non-supported regions (weighted sum cannot reach them, common under fixed-charge structure) and weight clustering (a finite grid re-discovering the same corners, even on a nearly convex frontier). The fill step treats both the same. + +If all boxes are small and even, the sweep is likely adequate — say so and stop. + +### Fill the largest gaps first + +For each flagged box, solve one ε-constraint subproblem targeted inside it: optimize one objective with the other bounded at the box midpoint (bi-objective; with more objectives, sort by each objective in turn and place one target per flagged box instead of recursing). Only certified `Optimal` results settle or steer anything here — a time-limited incumbent is kept as a point (tagged, below) but proves nothing about the gap. A new certified point that survives Step 4's dominance filter means the gap was real (an ε solve can return a weakly optimal point) — bisect: two more targets inside the two sub-boxes it creates. A certified endpoint coming back clears just the probed side of the bound; certifying the whole box as a true discontinuity also needs a known objective step size — all-integer objective coefficients over integer variables give one — to place the bound just inside the far endpoint and match its certified optimum. Without that step size, report the box as a candidate gap, not a proven discontinuity. Stop on a solve budget, or when the remaining boxes fall below the flag bar. + +### Warm-start each solve (cheap insurance) + +Consecutive fill solves differ by one bound, so seed each with its neighbor as a MIP start (Step 4's warm-start note) — one line, and it never changes what is optimal. Expect unchanged solve times; the value is insurance on hard subproblems. + +### Degrade gracefully, never silently + +If a subproblem hits its time limit with a feasible incumbent (`FeasibleFound`), keep the point — it is feasible, and the solve's reported gap bounds its suboptimality — but record it as approximate. The time-capped solve is the primary fallback: it already runs cuOpt's GPU heuristics alongside branch-and-bound. Heuristics-only mode (`mip_heuristics_only`) drops the proof work and returns feasible points with no gap bound — use it when feasible points are all you need, and tag everything it returns approximate. + +### Report with provenance + +Every presented point carries one of two tags: + +- **exact** — `Optimal` at your gap setting, i.e. optimal to that gap (Step 4); +- **approximate** — time-limited incumbent (quote its reported gap) or heuristics-only result (no bound exists; say so). + +State the counts with the frontier ("14 points, 11 exact, 3 approximate near the low-cost end, worst gap 2.4%"). Never present a mixed frontier as uniformly optimal. + +## Step 6 — interpret the frontier - **Report tradeoffs, not single numbers.** A frontier point means nothing in isolation. Quote the exchange rate — "≈ $4k of extra cost per 1% of added coverage in this region" — so the user can judge whether a move is worth it. On an LP/QP frontier this exchange rate is the swept constraint's dual at that point — the local slope of the frontier, accurate to the solve's optimality tolerance (tighten it before relying on a dual); on MILP, estimate it from the gap to the adjacent frontier point. - **Flag knee points; don't auto-pick them.** The "knee" is where the curve bends most sharply — beyond it you pay a lot for a little. It's often the best-balanced compromise and worth highlighting, but the final choice is the user's preference, not a rule. At the knee the slope is two-sided — the dual just below differs from just above — so quote the exchange rate there as a range, not one number. -- **Treat dominated or gappy output as a diagnostic.** If dominated points survive filtering, or the frontier is implausibly sparse or perfectly linear, suspect the sweep or the model — most often weighted-sum hiding a concave region (switch to ε-constraint) or a normalization mistake. +- **Treat dominated or gappy output as a diagnostic.** If dominated points survive filtering, or the frontier is implausibly sparse or perfectly linear, suspect the sweep or the model — most often weighted-sum hiding a concave region (return to Step 5 and fill the gaps) or a normalization mistake. - **State the weighting/ε you used.** Every reported point is conditional on its scalarization. Make that explicit so a single solve is never mistaken for "the" optimum. On LP/QP, the ε-constraint duals are the *implicit weights* at that point — the effective price the solution puts on each constrained objective, and the weights a weighted-sum solve would need to reproduce that tradeoff. Reporting them makes the accepted tradeoff ratio explicit. ## Interfaces diff --git a/skills/cuopt-multi-objective-exploration/evals/evals.json b/skills/cuopt-multi-objective-exploration/evals/evals.json index 44b2620185..e2a4922c94 100644 --- a/skills/cuopt-multi-objective-exploration/evals/evals.json +++ b/skills/cuopt-multi-objective-exploration/evals/evals.json @@ -68,5 +68,45 @@ "Reports supply in interpretable units, flags the knee, names candidate operating points, and defers the budget call to leadership", "Distinguishes this from a hard-budget case (cf. the decoy): cost is unconstrained here, so it exposes the tradeoff rather than returning a single plan" ] + }, + { + "id": "multiobj-explore-eval-006-gap-detection-completion", + "question": "A planner swept 20 weight combinations over cost and coverage for a facility-selection MILP and got only 6 distinct plans, with a big empty stretch in the middle of the curve. She asks: 'Are there really no options in between, or is my sweep missing them?' Advise her.", + "expected_skill": "cuopt-multi-objective-exploration", + "expected_script": null, + "ground_truth": "The agent recognizes the symptom of a weighted-sum sweep on a MILP: weighted-sum reaches only supported points (the convex hull of the frontier), so non-supported plans in between are invisible no matter how many weights are tried. It does NOT recommend simply adding more weight vectors. It measures the miss by forming the boxes between adjacent swept points, treats boxes much larger than the median spacing as candidate gaps, and fills the largest first with targeted epsilon-constraint subproblems (optimize one objective with the other bounded inside the box, bisecting when a new point appears). It notes that only certified Optimal solves settle a gap (a time-limited incumbent proves nothing), that a probe returning an existing endpoint clears just the probed side of the bound, and that certifying a true discontinuity additionally needs a known objective step size (e.g. all-integer objective coefficients over integer variables) to place a bound just inside the far endpoint -- otherwise the region is reported as a candidate gap, not a proven one -- and that filling stops on a solve budget or when the remaining boxes are small. It defers per-solve mechanics to the api skills and formulation to cuopt-numerical-optimization-formulation.", + "expected_behavior": [ + "Attributes the empty stretch to weighted-sum reaching only supported (convex-hull) points on a MILP, not to the intermediate options being absent", + "Does NOT prescribe more weight vectors as the fix", + "Measures the miss via the boxes between adjacent swept points and flags outsized boxes as candidate gaps", + "Fills the largest gaps first with targeted epsilon-constraint subproblems bounded inside each box, bisecting when new points appear", + "Does NOT declare a gap empty from a single midpoint probe; certifies a discontinuity only with certified Optimal solves plus a known objective step size (bound just inside the far endpoint), otherwise reports a candidate gap; stops on a solve budget or size threshold" + ] + }, + { + "id": "multiobj-explore-eval-007-provenance-reporting", + "question": "While tracing a cost-vs-emissions frontier for a plant-scheduling MILP with per-solve time limits, three of an analyst's epsilon-constraint solves hit their limit but returned feasible solutions. He asks: 'Can I include those points in the tradeoff table I hand my team?' Advise him.", + "expected_skill": "cuopt-multi-objective-exploration", + "expected_script": null, + "ground_truth": "The agent says yes, keep the time-limited incumbents — they are feasible and near the frontier — but only with provenance: every point in the table is tagged exact (proved optimal for its scalarization at the gap setting) or approximate (time-limited incumbent or heuristics-only result), each time-limited incumbent quoting its solver-reported remaining gap or bound (a heuristics-only point has no bound, which is itself stated), and the two counts are reported alongside the frontier (e.g. '14 points, 11 exact, 3 approximate near the low-cost end, worst gap 2%'). The mixed frontier is never presented as uniformly optimal; silently including the points and discarding them are both wrong. Optionally it notes that a persistently slow region can be populated with a short heuristics-only run (also tagged approximate) and that a larger per-solve budget can later upgrade approximate points to exact.", + "expected_behavior": [ + "Keeps the feasible time-limited incumbents rather than discarding them", + "Tags every reported point exact or approximate and reports the two counts with the frontier", + "Never presents the mixed frontier as uniformly optimal; approximate points are identified, not silently included", + "Distinguishes proved-optimal-at-gap (exact) from time-limited or heuristics-only results (approximate), quoting each incumbent's reported gap or bound and noting heuristics-only points carry none", + "Leaves the tradeoff choice to the team; the table shows options, not one answer" + ] + }, + { + "id": "multiobj-explore-eval-008-single-objective-lp-decoy", + "question": "A plant manager wants next month's production schedule at MINIMUM total cost, subject to machine capacity, labor hours, and contracted demand — all firm requirements. Report the optimal schedule and its cost.", + "expected_skill": null, + "expected_script": null, + "ground_truth": "DECOY (negative) — the multi-objective-exploration skill should NOT activate, and in particular the front-completion workflow should not. There is a single clear objective (minimize cost) and the requirements are genuinely fixed, so there is no tradeoff to explore, no frontier to trace, and no gaps to measure or fill. The correct response is one optimization solve returning ONE schedule and its cost.", + "expected_behavior": [ + "Recognizes a single objective with hard constraints, not a tradeoff", + "Does NOT trace a frontier, sweep weights or epsilon bounds, or run any gap-measurement/fill pass", + "Returns one optimal schedule and its cost from one solve" + ] } ]