Add front-completion workflow to cuopt-multi-objective-exploration skill#1550
Open
cafzal wants to merge 2 commits into
Open
Add front-completion workflow to cuopt-multi-objective-exploration skill#1550cafzal wants to merge 2 commits into
cafzal wants to merge 2 commits into
Conversation
cafzal
force-pushed
the
skill-multiobj-front-completion
branch
6 times, most recently
from
d2240b7 to
0d23d81
Compare
Signed-off-by: cafzal <cameron.afzal@gmail.com>
cafzal
force-pushed
the
skill-multiobj-front-completion
branch
from
0d23d81 to
e202579
Compare
Contributor
Author
|
@ramakrishnap-nv I realized this might be a helpful follow-up to the multi-objective skill. Small addition so I opened a PR directly but happy to convert to an issue instead if it warrants further discussion. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughChangesMulti-objective frontier completion
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/cuopt-multi-objective-exploration/evals/evals.json`:
- Around line 87-99: Update the ground_truth and expected_behavior for
multiobj-explore-eval-007-provenance-reporting to require reporting each
approximate incumbent’s solver-provided absolute or relative optimality
gap/bound, alongside its exact/approximate provenance label and frontier counts.
Ensure the guidance distinguishes proved-optimal points from time-limited or
heuristics-only results and does not accept approximate labels without
quantitative quality information.
- Around line 100-110: Update the expected_behavior entry for
multiobj-explore-eval-008-single-objective-lp-decoy to remove the mandatory
requirement to cite binding constraints, or make that analysis explicitly
optional; retain the requirements to recognize a single hard-constrained
objective, avoid multi-objective workflows, and return one optimal schedule with
its cost.
- Around line 77-83: Update the ground_truth and expected_behavior in the
evaluation oracle to make “just inside the far endpoint” certification
conditional on known objective spacing and a certified solve; otherwise require
describing the region only as a candidate gap, not a confirmed MILP
discontinuity. Preserve the existing requirements about supported weighted-sum
points, box-based gap detection, targeted epsilon-constraint filling, and
solve-budget or size-threshold stopping.
In `@skills/cuopt-multi-objective-exploration/SKILL.md`:
- Line 138: Revise the discontinuity-certification guidance to require a known,
justified objective lattice step and a certified optimal solve before moving a
bound “just inside” the far endpoint. Do not infer integer objective values
solely from integer input data when continuous variables exist; without both
prerequisites, label the result a candidate discontinuity rather than a proof.
- Line 138: Clarify the ε-constraint fill recursion for the multi-objective
workflow: either explicitly scope the midpoint bisection, endpoint clearing, and
certification procedure to the bi-objective case, or define how k > 2 probing
partitions the box into all unexplored orthants and queues each subregion for
filling. Update the surrounding flagged-box and dominance-filter guidance so it
consistently follows the selected scope.
- Around line 150-155: Update the frontier reporting guidance to replace “exact”
with “certified-to-gap” for points returned under a nonzero MIP gap, while
retaining “approximate” for time-limited or heuristic results. Require each
point to include its numeric absolute or relative gap, or incumbent-versus-bound
values, in addition to aggregate counts; revise the “Every presented point” tags
and example wording accordingly.
- Around line 132-138: Clarify the gap-filling workflow so sampled dominance is
never treated as a frontier certificate. Distinguish feasible or weakly optimal
results from verified Pareto points: retain feasible points only as probes,
exclude them from discontinuity certification, and require an independently
verified Pareto/optimal result before certifying a box or using it to guide
recursive bisection.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 4a95d743-80b2-4b61-980b-707a387372f0
📒 Files selected for processing (2)
skills/cuopt-multi-objective-exploration/SKILL.mdskills/cuopt-multi-objective-exploration/evals/evals.json
Comment on lines
+150
to
+155
| Every presented point carries one of two tags: | ||
|
|
||
| - **exact** — `Optimal` at your gap setting, i.e. optimal to that gap (Step 4); | ||
| - **approximate** — time-limited incumbent or heuristics-only result. | ||
|
|
||
| State the counts with the frontier ("14 points, 11 exact, 3 approximate near the low-cost end"). Never present a mixed frontier as uniformly optimal. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Separate “optimal-at-gap” from exact, and report the numeric bound.
A nonzero MIP gap is not mathematically exact, yet line 152 labels that result exact. Use terminology such as certified-to-gap, and include each approximate point’s absolute/relative gap or incumbent-versus-bound value rather than only reporting counts. (docs.nvidia.com)
Suggested provenance wording
-- exact — `Optimal` at your gap setting
+- certified-to-gap — `Optimal` under the configured gap tolerance;
+ report that tolerance with the point
- approximate — time-limited incumbent or heuristics-only result
+- approximate — time-limited or heuristics-only result; report its
+ absolute/relative bound when available📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| Every presented point carries one of two tags: | |
| - **exact** — `Optimal` at your gap setting, i.e. optimal to that gap (Step 4); | |
| - **approximate** — time-limited incumbent or heuristics-only result. | |
| State the counts with the frontier ("14 points, 11 exact, 3 approximate near the low-cost end"). Never present a mixed frontier as uniformly optimal. | |
| Every presented point carries one of two tags: | |
| - **certified-to-gap** — `Optimal` under the configured gap tolerance; report that tolerance with the point | |
| - **approximate** — time-limited or heuristics-only result; report its absolute/relative bound when available | |
| State the counts with the frontier ("14 points, 11 exact, 3 approximate near the low-cost end"). Never present a mixed frontier as uniformly optimal. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/cuopt-multi-objective-exploration/SKILL.md` around lines 150 - 155,
Update the frontier reporting guidance to replace “exact” with
“certified-to-gap” for points returned under a nonzero MIP gap, while retaining
“approximate” for time-limited or heuristic results. Require each point to
include its numeric absolute or relative gap, or incumbent-versus-bound values,
in addition to aggregate counts; revise the “Every presented point” tags and
example wording accordingly.
…gaps Signed-off-by: cafzal <cameron.afzal@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The skill teaches weighted-sum vs ε-constraint scalarization and stops at "ε-constraint reaches non-supported points." This adds the operational sequel as a new Step 5 — complete the frontier: measure what a sweep missed (the boxes between adjacent swept points), fill the largest gaps first with targeted ε solves, keep time-limited feasible incumbents with their gap as the suboptimality bound (heuristics-only mode covers the no-certificates-needed case), and tag every point exact vs approximate so a mixed frontier is never over-claimed. Adds no solver features; orchestrates shipped APIs (MIP starts, time limits, heuristics-only mode). The interpret section is renumbered to Step 6.
Motivation, measured against exactly enumerated ground-truth fronts on a fixed-cost selection MILP family (median over 10 seeds × 3 instances, n = 60–240): the supported points any weighted-sum sweep can reach cap at 0.82–0.90 of front hypervolume — more weights cannot close that — while the same budget on targeted ε solves reaches ~0.99. On a near-convex variant the miss persists (weight grids cluster at corners), so measure-then-fill pays regardless of the front's convexity.
Verified end to end with cuOpt 26.06.00 on a T4, driving the section exactly as written on models adapted from cuopt-examples:
Optimalsolves — hypervolume 0.489The p-median run also exercised the degrade path for real: time-capped solves returned
FeasibleFoundincumbents, kept and tagged approximate, and heuristics-only mode populated the same subproblem.The verification loop also caught and fixed one defect in the measure: a sweep that returns only a handful of points is all gaps, so no box stands out from the median — the text now also flags boxes covering a large share of the frontier's span.
Evals: +2 positive (gap detection and fill; provenance reporting), +1 negative (single-objective decoy), mirroring the existing schema. Setting and status names verified in source and docs on this branch;
validate_skills.shpasses;BENCHMARK.md/ skill card / signature regenerate via NVSkills CI. Benchmark harness and per-seed tables available on request.Checklist