Skip to content

Conversation

@kaiming-cheng
Copy link
Contributor

This PR adds bottleneck analysis prompt building and response parsing to the diagnose module.

Core Components

1. BottleneckResult (judger_prompt.py)

Single dataclass representing a bottleneck analysis:

  • Category: memory, compute, or underutilized
  • Summary: one-line description
  • Reasoning: explanation citing metrics
  • Root_causes: list of causes with metric evidence
  • Recommended_fixes: actionable fixes with rationale
  • Configurable analysis:
    • num_bottlenecks: how many bottlenecks to identify (default: 2)
    • num_causes: root causes per bottleneck (default: 2)
    • num_fixes: fixes per bottleneck (default: 1)

2. Prompt Builder (judger_prompt.py)

Constructs structured LLM prompts from:

  • Kernel source code
  • NCU profiling metrics (formatted via metric_schema)
  • Roofline analysis results (via ncu_roofline)
  • GPU hardware specifications (via gpu_spec)

Example Usage

  prompt = build_bottleneck_prompt(                                                                                             
      kernel_code=kernel_src,                                                                                                   
      ncu_metrics=ncu_data,                                                                                                     
      roofline=roofline_result,                                                                                                 
      gpu_specs=gpu_specs,                                                                                                      
      num_bottlenecks=2,                                                                                                        
      num_causes=2,                                                                                                             
      num_fixes=1,                                                                                                              
  )                                                                                                                             
                                                                                                                                
  # After LLM call...              
   results = parse_bottleneck_response(llm_response)                     

More end-to-end testing in future PR

Kaiming Cheng and others added 30 commits January 15, 2026 11:44
Consolidates previous kernel_benchmark.py and pytorch_benchmark.py into a
streamlined 3-file architecture with clear separation of concerns:

Architecture:
- benchmark.py (299 lines): Main Benchmark class with simplified API
  - benchmark_kernel(): Always uses subprocess for crash protection
  - benchmark_pytorch(): Always uses direct mode for stable code
  - BenchmarkLockManager: GPU lock management for multi-worker scenarios

- timing.py (437 lines): Complete timing infrastructure
  - Timing: time_with_cuda_events(), time_with_triton_do_bench()
  - Loading: prepare_pytorch_model(), load_kernel_function()
  - Stats: compute_timing_stats() with essential metrics (mean/std/min/max)

- kernel_subprocess.py (442 lines): Subprocess runner for kernel isolation
  - Crash protection for potentially buggy kernels
  - Clean CUDA state between runs
  - Timeout handling

Key improvements:
- Eliminated string code generation (was generating Python as strings)
- Removed unnecessary statistics (median, p25/p75/p95/p99)
- Removed confusing use_subprocess parameter (behavior now deterministic)
- Fixed dtype bug causing incorrect speedup measurements
- Reduced from 5 files to 3 files with clearer naming
- Code reduction: ~1,400 lines → 1,178 lines

Simple API:
  bench = Benchmark(logger, temp_dir, lock, worker_id)
  pytorch_result = bench.benchmark_pytorch(problem_file)
  kernel_result = bench.benchmark_kernel(kernel_file, problem_file)
  speedup = pytorch_result['stats']['mean'] / kernel_result['time_ms']
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jan 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants