Issues

• [Bug] TGI/Triton/MindIE 服务化后端不解析 details.generated_tokens，性能报告 Token Throughput 恒为 0

  content_check_passed
  AISBench/benchmark#616 · Filter by author lsyx6399 opened on Oct 28, 2026
  2

• [Bug] humaneval数据集精度评测报错

  bugcontent_check_failed
  AISBench/benchmark#613 · Filter by author Xiaoyue-X opened on Oct 27, 2026
  4

• [Feature]: 新增 SimpleQA Verified 数据集适配

  content_check_passed
  AISBench/benchmark#612 · Filter by author llauren48 opened on Oct 26, 2026
  5

• [RFC][AISBench]支持VABench音视频联合生成模型精度测评

  content_check_failed
  AISBench/benchmark#611 · Filter by author Libotry opened on Oct 26, 2026
  3

• [Feature]: Add OpenAI Responses API Support for LLM Serving Benchmarking

  content_check_passedfeature
  AISBench/benchmark#610 · Filter by author mze994667-jpg opened on Oct 24, 2026
  2

• [易用性] 测试url时没有host_ip和host_port也必须要填

  bugcontent_check_passed
  AISBench/benchmark#609 · Filter by author bzncu opened on Oct 23, 2026
  3

• [Roadmap] AISBench 2026 Q4 Roadmap

  AISBench/benchmark#608 · Filter by author Libotry opened on Oct 21, 2026
  2

• [Bug] 稳态重算（Stable Stage）Output/Total Token Throughput 分子分母时间不对齐，吞吐被高估约 27×

  bugcontent_check_passed
  AISBench/benchmark#605 · Filter by author DoubleEspresso-7 opened on Oct 20, 2026
  2

• [issue template bug] issue模板显示异常

  content_check_failednot_fixed
  AISBench/benchmark#602 · Filter by author SJTUyh opened on Oct 16, 2026
  5

• [Bug] VBench数据集 standard测评模式异常

  bugcontent_check_passedin progress
  AISBench/benchmark#598 · Filter by author socrahow opened on Oct 16, 2026 · AISBench 2026 Q4 Roadmap
  5

  

• [疑问] VLLM 流式响应未完成时被误标为成功（正常 HTTP EOF 缺少 finish_reason）

  content_check_passedpendingquestion
  AISBench/benchmark#593 · Filter by author mantuoluozk opened on Oct 13, 2026 · AISBench 2026 Q4 Roadmap
  6

  

• [Bug] 严格 seccomp 容器环境下因无法创建线程导致 tokenizers panic 与 ThreadPoolExecutor 崩溃，缺少串行回退机制

  content_check_failedother
  AISBench/benchmark#588 · Filter by author student-jhz opened on Oct 10, 2026 · AISBench 2026 Q4 Roadmap
  5

• [需求] 建议支持https://<ip:port>/v2/xxxxxxxxxxxx/v1/chat/completions格式的大模型接口

  content_check_passedenhancement需求讨论
  AISBench/benchmark#584 · Filter by author yaoyanxiang opened on Sep 30, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] ais_bench_prefix_cache不支持sglang metrics，不支持sglang:prefill_effective_tokens_total等参数

  content_check_passedenhancement需求讨论
  AISBench/benchmark#583 · Filter by author Royce-sun opened on Sep 29, 2026 · AISBench 2026 Q4 Roadmap
  5

  

• [疑问] [LiveCodeBench v6] 判题器不支持 sys.stdout.buffer，可能将正确程序判为运行错误

  content_check_passed
  AISBench/benchmark#579 · Filter by author mantuoluozk opened on Sep 29, 2026 · AISBench 2026 Q4 Roadmap
  8

• [疑问] [LiveCodeBench v6] 部分正确答案因非法测试用例或单一参考输出被误判

  content_check_passed
  AISBench/benchmark#576 · Filter by author mantuoluozk opened on Sep 28, 2026 · AISBench 2026 Q4 Roadmap
  7

• [需求] 希望在最后的统计看板体现request rate

  content_check_passedenhancement需求讨论
  AISBench/benchmark#540 · Filter by author qiudepei opened on Sep 14, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 多模态解析的测试用例报告，需要自动解析多模态文件的分辨率参数

  content_check_passedenhancement需求讨论
  AISBench/benchmark#539 · Filter by author Happystone78 opened on Sep 14, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [主动优化] Agent数据集结果可视化解析+Badcase分类归因

  content_check_passedenhancement需求讨论
  AISBench/benchmark#523 · Filter by author ivanbao9783 opened on Sep 8, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 希望能够支持现网流量会放的方式验证agent场景推理的处理能力

  content_check_failedenhancementin progress需求讨论
  Task

  AISBench/benchmark#498 · Filter by author chenghaitao1987 opened on Aug 26, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [enhancement] 虚拟内存占用检测是不是按照大小检测 而不是百分比

  content_check_passedenhancement需求讨论
  AISBench/benchmark#480 · Filter by author zhongzhouTan-coder opened on Aug 20, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 支持量化场景下的压缩测评集快速精度评估能力

  content_check_passedenhancement需求讨论
  AISBench/benchmark#475 · Filter by author flame-hu opened on Aug 18, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] aisbench 支持 I2V视频生成的评测

  content_check_passedenhancement需求讨论
  AISBench/benchmark#470 · Filter by author ethan686 opened on Aug 17, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] agent驱动测评

  content_check_passedenhancement需求讨论
  AISBench/benchmark#435 · Filter by author joejoezrc opened on Aug 4, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求]

  content_check_passedenhancement需求讨论
  AISBench/benchmark#434 · Filter by author joejoezrc opened on Aug 4, 2026 · AISBench 2026 Q4 Roadmap
  5