Fun-ASR-Nano-2512离线 SDK 推理的示例代码没有识别出语音结果。

[Chaiyym]

Before asking

1. Search existing issues: https://github.com/modelscope/FunASR/issues
2. Search the docs: https://modelscope.github.io/FunASR/
3. Check the README quick start and deployment section.

Question

运行FunASR官网中离线 SDK 推理的示例代码没有识别出语音结果。

#测试代码
from funasr.auto.auto_model_vllm import AutoModelVLLM
def test():
    model = AutoModelVLLM(
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        hub="ms",  # 也可以使用 "hf"
        tensor_parallel_size=1,
        gpu_memory_utilization=0.8,
        dtype="fp16"
    )
    results = model.generate(
        ["/data/wh/dev/FunASR/runtime/triton_gpu/client/test_wavs/long.wav"],
        language="中文",
        hotwords=["张三", "北京"],
    )
    for item in results:
        print(f"[{item}]")
        print(f"[{item['key']}] {item['text']}")
if __name__ == "__main__":
    test()

#运行命令
CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py

# 启动日志
(FunASR) ➜  FunASR git:(main) ✗ CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
2026-06-16 00:36:45,686 [INFO] vLLM model already prepared at /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm
2026-06-16 00:36:47,332 [INFO] Loading audio component weights from /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/model.pt
2026-06-16 00:36:49,279 [INFO]   Loaded audio_encoder: 914 params
2026-06-16 00:36:49,285 [INFO]   Loaded audio_adaptor: 36 params
2026-06-16 00:36:49,930 [INFO] Initializing vLLM with model: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm
2026-06-16 00:36:49,931 [INFO]   tensor_parallel_size=1
2026-06-16 00:36:49,931 [INFO]   gpu_memory_utilization=0.8
INFO 06-16 00:36:49 [utils.py:233] non-default args: {'enable_prompt_embeds': True, 'trust_remote_code': True, 'dtype': 'float16', 'max_model_len': 4096, 'gpu_memory_utilization': 0.8, 'disable_log_stats': True, 'model': '/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm'}
INFO 06-16 00:36:49 [model.py:549] Resolved architecture: Qwen3ForCausalLM
WARNING 06-16 00:36:49 [model.py:2016] Casting torch.bfloat16 to torch.float16.
INFO 06-16 00:36:49 [model.py:1678] Using max model len 4096
INFO 06-16 00:36:49 [scheduler.py:238] Chunked prefill is enabled with max_num_batched_tokens=8192.
INFO 06-16 00:36:49 [vllm.py:790] Asynchronous scheduling is enabled.
WARNING 06-16 00:36:51 [system_utils.py:152] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore pid=32072) INFO 06-16 00:37:00 [core.py:105] Initializing a V1 LLM engine (v0.19.1) with config: model='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', speculative_config=None, tokenizer='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_images_per_batch': 0, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []}
(EngineCore pid=32072) INFO 06-16 00:37:01 [parallel_state.py:1400] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:39793 backend=nccl
(EngineCore pid=32072) INFO 06-16 00:37:01 [parallel_state.py:1716] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=32072) INFO 06-16 00:37:02 [gpu_model_runner.py:4735] Starting to load model /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm...
(EngineCore pid=32072) ERROR 06-16 00:37:02 [fa_utils.py:145] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore pid=32072) INFO 06-16 00:37:02 [cuda.py:334] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION'].
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.33it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.33it/s]
(EngineCore pid=32072) 
(EngineCore pid=32072) INFO 06-16 00:37:03 [default_loader.py:384] Loading weights took 0.44 seconds
(EngineCore pid=32072) INFO 06-16 00:37:04 [gpu_model_runner.py:4820] Model loading took 1.12 GiB memory and 0.992266 seconds
(EngineCore pid=32072) INFO 06-16 00:37:07 [backends.py:1051] Using cache directory: /home/xty/.cache/vllm/torch_compile_cache/fe9c261fb7/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=32072) INFO 06-16 00:37:07 [backends.py:1111] Dynamo bytecode transform time: 3.07 s
(EngineCore pid=32072) INFO 06-16 00:37:09 [backends.py:285] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 0.973 s
(EngineCore pid=32072) INFO 06-16 00:37:09 [decorators.py:305] Directly load AOT compilation from path /home/xty/.cache/vllm/torch_compile_cache/torch_aot_compile/f4c695bdd811bfcaa114b11fad47b26ba0df24f513a51e5635e73f77d4ca426e/rank_0_0/model
(EngineCore pid=32072) INFO 06-16 00:37:09 [monitor.py:48] torch.compile took 4.72 s in total
(EngineCore pid=32072) INFO 06-16 00:37:09 [monitor.py:76] Initial profiling/warmup run took 0.08 s
(EngineCore pid=32072) INFO 06-16 00:37:09 [kv_cache_utils.py:829] Overriding num_gpu_blocks=0 with num_gpu_blocks_override=512
(EngineCore pid=32072) INFO 06-16 00:37:09 [gpu_model_runner.py:5876] Profiling CUDA graph memory: PIECEWISE=51 (largest=512), FULL=35 (largest=256)
(EngineCore pid=32072) INFO 06-16 00:37:11 [gpu_model_runner.py:5955] Estimated CUDA graph memory: 0.47 GiB total
(EngineCore pid=32072) INFO 06-16 00:37:11 [gpu_worker.py:436] Available KV cache memory: 23.55 GiB
(EngineCore pid=32072) INFO 06-16 00:37:11 [gpu_worker.py:470] In v0.19, CUDA graph memory profiling will be enabled by default (VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1), which more accurately accounts for CUDA graph memory during KV cache allocation. To try it now, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1 and increase --gpu-memory-utilization from 0.8000 to 0.8150 to maintain the same effective KV cache size.
(EngineCore pid=32072) INFO 06-16 00:37:11 [kv_cache_utils.py:1319] GPU KV cache size: 220,448 tokens
(EngineCore pid=32072) INFO 06-16 00:37:11 [kv_cache_utils.py:1324] Maximum concurrency for 4,096 tokens per request: 53.82x
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [00:01<00:00, 25.76it/s]
Capturing CUDA graphs (decode, FULL): 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 27.55it/s]
(EngineCore pid=32072) INFO 06-16 00:37:15 [gpu_model_runner.py:6046] Graph capturing finished in 4 secs, took 0.42 GiB
(EngineCore pid=32072) INFO 06-16 00:37:15 [gpu_worker.py:597] CUDA graph pool memory: 0.42 GiB (actual), 0.47 GiB (estimated), difference: 0.06 GiB (13.6%).
(EngineCore pid=32072) INFO 06-16 00:37:15 [core.py:283] init engine (profile, create kv cache, warmup model) took 11.46 seconds
(EngineCore pid=32072) INFO 06-16 00:37:16 [vllm.py:790] Asynchronous scheduling is enabled.
2026-06-16 00:37:17,681 [INFO] Loaded embedding layer: torch.Size([151936, 1024])
2026-06-16 00:37:17,956 [INFO] Audio encoding: 1 samples in 0.255s
2026-06-16 00:37:19,927 [INFO] vLLM generation: 1.971s
[{'key': 'long', 'text': ''}]
[long] 
(EngineCore pid=32072) INFO 06-16 00:37:20 [core.py:1210] Shutdown initiated (timeout=0)
(EngineCore pid=32072) INFO 06-16 00:37:20 [core.py:1233] Shutdown complete
[rank0]:[W616 00:37:20.703451303 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

Environment

• OS: ubuntu
• FunASR version: master
• Device (cuda, cpu, mps): cuda
• GPU model: v100 32G
  Python 3.12.13
  vllm 0.19.1
  torch 2.10.0+cu126
  torch-c-dlpack-ext 0.1.5
  torch-complex 0.4.4
  torchaudio 2.10.0+cu126
  torchvision 0.25.0+cu126
  cuda-bindings 12.9.4
  cuda-core 1.0.1
  cuda-pathfinder 1.5.5
  cuda-python 12.9.4
  cuda-tile 1.3.0
  cuda-toolkit 12.6.3
  nvidia-cuda-cccl 13.3.3.3.1
  nvidia-cuda-crt 13.3.33
  nvidia-cuda-cupti 13.0.85
  nvidia-cuda-cupti-cu12 12.6.80
  nvidia-cuda-nvcc 13.2.78
  nvidia-cuda-nvrtc 13.3.33
  nvidia-cuda-nvrtc-cu12 12.6.77
  nvidia-cuda-runtime 13.3.29
  nvidia-cuda-runtime-cu12 12.6.77
  nvidia-cuda-tileiras 13.2.78

[LauraGPT]

感谢反馈！看你贴的「测试代码」和「运行命令」其实是两个不同的东西，问题很可能出在这里：

1. 命令和脚本对不上
你贴的运行命令是 serve_realtime_ws.py——它启动的是实时 WebSocket 服务，不会直接打印离线识别结果；而你贴的那段 test() 离线推理脚本其实并没有被执行。请把 test() 单独存成一个文件直接运行：

# 存成 test_offline.py 后
CUDA_VISIBLE_DEVICES=7 python test_offline.py

2. vLLM 版本
Fun-ASR-Nano 的 vLLM 推理对版本比较敏感，建议 vLLM 0.12.0 + torch 2.9.0（参考 #2968）。版本不匹配时常表现为「能跑但输出为空」。

3. 几个排查点

• 音频路径 long.wav 在你机器上是否真实存在、且为 16k 单声道 wav？
• 模型是否下载完整：hub="ms" 走 ModelScope，网络不稳可能只下了部分文件，可换 hub="hf" 试；
• 看完整控制台日志，有没有 vLLM 的 warning/报错。

方便的话，把直接运行 test() 脚本后的完整输出贴上来（含加载日志和 print(item) 的内容），我帮你定位。如果是文档示例本身的问题，我们会修正 vllm.html 的示例。🙏

[Chaiyym]

单独提取出了测试内容，并使用了同一个语音文件对比了AutoModel和AutoModelVLLM生成结果。
1.排除了音频文件和模型文件可能的问题
2.AutoModel接口可以正常输出识别内容
3.AutoModelVLLM无输出识别内容

1.测试脚本

import time
from funasr import AutoModel
from funasr.auto.auto_model_vllm import AutoModelVLLM

local_file="/data/wh/dev/FunASR/asr_example_zh.wav"

# AutoModel接口测试
def auto_model_test():
    model = AutoModel(
        hub="ms",
        device="cuda:0",
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        vad_model="fsmn-vad",
        vad_kwargs={"max_single_segment_time": 30000},
        trust_remote_code=True,
    )
    res = model.generate(input=local_file, cache={}, batch_size=1,  hotwords=["keyword"], language="中文")
    for item in res:
        print(f"[{item}]")
    time.sleep(12)

# AutoModelVLLM接口测试
def auto_model_vllm_test():
    model = AutoModelVLLM(
        hub="ms",
        device="cuda:0",
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        vad_model="fsmn-vad",
        vad_kwargs={"max_single_segment_time": 30000},
        enforce_eager=True,
        trust_remote_code=True,
        tensor_parallel_size=1,
        gpu_memory_utilization=0.8,
        dtype="fp16",

    )
    res = model.generate(inputs=local_file, cache={}, batch_size=1, hotwords=["keyword"], language="中文")
    for item in res:
        print(f"[{item}]")
    time.sleep(120)

if __name__ == "__main__":
    auto_model_test()
    print("---------------------------------------")
    auto_model_vllm_test()

2.完整的运行命令

(FunASR) ➜  FunASR git:(main) ✗ CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/test.py 
funasr version: 1.3.9.
Check update of funasr, and it would cost few times. You may disable it by set `disable_update=True` in AutoModel
You are using the latest version of funasr-1.3.9
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
WARNING:root:trust_remote_code: True
Loading remote code successfully: model
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch
WARNING:root:trust_remote_code: False
rtf_avg: 0.018: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 10.13it/s]
rtf_avg: 0.169: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.20it/s]
rtf_avg: 0.151, time_speech:  5.547, time_escape: 0.836: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.19it/s]
[{'key': 'asr_example_zh', 'text': '欢迎大家来体验达摩院推出的语音识别模型。', 'text_tn': '欢迎大家来体验达摩院推出的语音识别模型', 'label': 'null', 'ctc_text': '欢迎大家来体验达摩院推出的语音识别模型', 'ctc_timestamps': [{'token': '欢', 'start_time': 0.42, 'end_time': 0.48, 'score': 0.999}, {'token': '迎', 'start_time': 0.6, 'end_time': 0.72, 'score': 0.993}, {'token': '大', 'start_time': 0.84, 'end_time': 0.9, 'score': 1.0}, {'token': '家', 'start_time': 0.96, 'end_time': 1.02, 'score': 1.0}, {'token': '来', 'start_time': 1.2, 'end_time': 1.26, 'score': 1.0}, {'token': '体', 'start_time': 1.5, 'end_time': 1.56, 'score': 0.998}, {'token': '验', 'start_time': 1.68, 'end_time': 1.74, 'score': 0.997}, {'token': '达', 'start_time': 1.92, 'end_time': 1.98, 'score': 0.982}, {'token': '摩', 'start_time': 2.1, 'end_time': 2.16, 'score': 0.201}, {'token': '院', 'start_time': 2.28, 'end_time': 2.34, 'score': 0.984}, {'token': '推', 'start_time': 2.52, 'end_time': 2.58, 'score': 0.997}, {'token': '出', 'start_time': 2.7, 'end_time': 2.82, 'score': 0.999}, {'token': '的', 'start_time': 2.94, 'end_time': 3.0, 'score': 1.0}, {'token': '语', 'start_time': 3.24, 'end_time': 3.3, 'score': 0.998}, {'token': '音', 'start_time': 3.42, 'end_time': 3.48, 'score': 0.995}, {'token': '识', 'start_time': 3.72, 'end_time': 3.78, 'score': 0.995}, {'token': '别', 'start_time': 3.9, 'end_time': 3.96, 'score': 0.996}, {'token': '模', 'start_time': 4.2, 'end_time': 4.26, 'score': 1.0}, {'token': '型', 'start_time': 4.38, 'end_time': 4.44, 'score': 0.998}], 'timestamps': [{'token': '欢', 'start_time': 1.03, 'end_time': 1.09, 'score': 0.999}, {'token': '迎', 'start_time': 1.21, 'end_time': 1.33, 'score': 0.993}, {'token': '大', 'start_time': 1.45, 'end_time': 1.51, 'score': 1.0}, {'token': '家', 'start_time': 1.57, 'end_time': 1.63, 'score': 1.0}, {'token': '来', 'start_time': 1.81, 'end_time': 1.87, 'score': 1.0}, {'token': '体', 'start_time': 2.11, 'end_time': 2.17, 'score': 0.998}, {'token': '验', 'start_time': 2.29, 'end_time': 2.35, 'score': 0.997}, {'token': '达', 'start_time': 2.53, 'end_time': 2.59, 'score': 0.982}, {'token': '摩', 'start_time': 2.71, 'end_time': 2.77, 'score': 0.201}, {'token': '院', 'start_time': 2.89, 'end_time': 2.95, 'score': 0.984}, {'token': '推', 'start_time': 3.13, 'end_time': 3.19, 'score': 0.997}, {'token': '出', 'start_time': 3.31, 'end_time': 3.43, 'score': 0.999}, {'token': '的', 'start_time': 3.55, 'end_time': 3.61, 'score': 1.0}, {'token': '语', 'start_time': 3.85, 'end_time': 3.91, 'score': 0.998}, {'token': '音', 'start_time': 4.03, 'end_time': 4.09, 'score': 0.995}, {'token': '识', 'start_time': 4.33, 'end_time': 4.39, 'score': 0.995}, {'token': '别', 'start_time': 4.51, 'end_time': 4.57, 'score': 0.996}, {'token': '模', 'start_time': 4.81, 'end_time': 4.87, 'score': 1.0}, {'token': '型', 'start_time': 4.99, 'end_time': 5.05, 'score': 0.998}, {'token': '。', 'start_time': 5.41, 'end_time': 5.47, 'score': 0.0}], 'timestamp': [[1030, 1090], [1210, 1330], [1450, 1510], [1570, 1630], [1810, 1870], [2110, 2170], [2290, 2350], [2530, 2590], [2710, 2770], [2890, 2950], [3130, 3190], [3310, 3430], [3550, 3610], [3850, 3910], [4030, 4090], [4330, 4390], [4510, 4570], [4810, 4870], [4990, 5050], [5410, 5470]]}]
---------------------------------------
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
INFO 06-16 13:48:03 [utils.py:233] non-default args: {'enable_prompt_embeds': True, 'trust_remote_code': True, 'dtype': 'float16', 'max_model_len': 4096, 'gpu_memory_utilization': 0.8, 'disable_log_stats': True, 'enforce_eager': True, 'model': '/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm'}
INFO 06-16 13:48:03 [model.py:549] Resolved architecture: Qwen3ForCausalLM
WARNING 06-16 13:48:03 [model.py:2016] Casting torch.bfloat16 to torch.float16.
INFO 06-16 13:48:03 [model.py:1678] Using max model len 4096
INFO 06-16 13:48:03 [scheduler.py:238] Chunked prefill is enabled with max_num_batched_tokens=8192.
INFO 06-16 13:48:03 [vllm.py:790] Asynchronous scheduling is enabled.
WARNING 06-16 13:48:03 [vllm.py:848] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
WARNING 06-16 13:48:03 [vllm.py:859] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
INFO 06-16 13:48:04 [vllm.py:1025] Cudagraph is disabled under eager mode
INFO 06-16 13:48:04 [compilation.py:292] Enabled custom fusions: norm_quant, act_quant
WARNING 06-16 13:48:05 [system_utils.py:152] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore pid=2378) INFO 06-16 13:48:13 [core.py:105] Initializing a V1 LLM engine (v0.19.1) with config: model='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', speculative_config=None, tokenizer='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_images_per_batch': 0, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []}
(EngineCore pid=2378) INFO 06-16 13:48:14 [parallel_state.py:1400] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:51727 backend=nccl
(EngineCore pid=2378) INFO 06-16 13:48:15 [parallel_state.py:1716] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=2378) INFO 06-16 13:48:15 [gpu_model_runner.py:4735] Starting to load model /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm...
(EngineCore pid=2378) ERROR 06-16 13:48:16 [fa_utils.py:145] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore pid=2378) INFO 06-16 13:48:16 [cuda.py:334] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION'].
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.42it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.42it/s]
(EngineCore pid=2378) 
(EngineCore pid=2378) INFO 06-16 13:48:16 [default_loader.py:384] Loading weights took 0.43 seconds
(EngineCore pid=2378) INFO 06-16 13:48:17 [gpu_model_runner.py:4820] Model loading took 1.12 GiB memory and 0.891017 seconds
(EngineCore pid=2378) INFO 06-16 13:48:19 [gpu_worker.py:436] Available KV cache memory: 23.56 GiB
(EngineCore pid=2378) INFO 06-16 13:48:19 [kv_cache_utils.py:1319] GPU KV cache size: 220,592 tokens
(EngineCore pid=2378) INFO 06-16 13:48:19 [kv_cache_utils.py:1324] Maximum concurrency for 4,096 tokens per request: 53.86x
(EngineCore pid=2378) INFO 06-16 13:48:19 [core.py:283] init engine (profile, create kv cache, warmup model) took 2.33 seconds
(EngineCore pid=2378) INFO 06-16 13:48:20 [vllm.py:790] Asynchronous scheduling is enabled.
(EngineCore pid=2378) WARNING 06-16 13:48:20 [vllm.py:848] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
(EngineCore pid=2378) WARNING 06-16 13:48:20 [vllm.py:859] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(EngineCore pid=2378) INFO 06-16 13:48:20 [vllm.py:1025] Cudagraph is disabled under eager mode
(EngineCore pid=2378) INFO 06-16 13:48:20 [compilation.py:292] Enabled custom fusions: norm_quant, act_quant
[{'key': 'asr_example_zh', 'text': ''}]
(EngineCore pid=2378) INFO 06-16 13:50:32 [core.py:1210] Shutdown initiated (timeout=0)
(EngineCore pid=2378) INFO 06-16 13:50:32 [core.py:1233] Shutdown complete
[rank0]:[W616 13:50:32.003651417 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

[Chaiyym]

再补充一下环境信息

依赖安装命令

uv venv --python=3.12
source .venv/bin/activate

uv pip install -e ./
uv pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --torch-backend=cu126
uv pip install safetensors tiktoken websockets regex fastapi uvicorn python-multipart soundfile
uv pip install vllm==0.19.1 --torch-backend=cu126

具体依赖版本

(FunASR) ➜  FunASR git:(main) ✗ uv pip list
Package                                  Version       Editable project location
---------------------------------------- ------------- -------------------------
aiohappyeyeballs                         2.6.2
aiohttp                                  3.14.1
aiosignal                                1.4.0
aliyun-python-sdk-core                   2.16.0
aliyun-python-sdk-kms                    2.16.5
annotated-doc                            0.0.4
annotated-types                          0.7.0
anthropic                                0.109.2
antlr4-python3-runtime                   4.9.3
anyio                                    4.14.0
apache-tvm-ffi                           0.1.12
astor                                    0.8.1
attrs                                    26.1.0
audioread                                3.1.0
blake3                                   1.0.8
cachetools                               7.1.4
cbor2                                    6.1.2
certifi                                  2026.5.20
cffi                                     2.0.0
charset-normalizer                       3.4.7
click                                    8.4.1
cloudpickle                              3.1.2
compressed-tensors                       0.15.0.1
crcmod                                   1.7
cryptography                             49.0.0
cuda-bindings                            12.9.4
cuda-pathfinder                          1.5.5
cuda-python                              12.9.4
cuda-toolkit                             12.6.3
decorator                                5.3.1
depyf                                    0.20.0
detect-installer                         0.1.0
dill                                     0.4.1
diskcache                                5.6.3
distro                                   1.9.0
dnspython                                2.8.0
docstring-parser                         0.18.0
editdistance                             0.8.1
einops                                   0.8.2
email-validator                          2.3.0
fastapi                                  0.137.1
fastapi-cli                              0.0.24
fastapi-cloud-cli                        0.20.0
fastar                                   0.11.0
filelock                                 3.29.4
flashinfer-cubin                         0.6.6
flashinfer-python                        0.6.6
frozenlist                               1.8.0
fsspec                                   2026.6.0
funasr                                   1.3.9         /data/wh/dev/FunASR
gguf                                     0.19.0
googleapis-common-protos                 1.75.0
grpcio                                   1.81.1
h11                                      0.16.0
hf-xet                                   1.5.1
httpcore                                 1.0.9
httptools                                0.8.0
httpx                                    0.28.1
httpx-sse                                0.4.3
huggingface-hub                          1.19.0
hydra-core                               1.3.3
idna                                     3.18
ijson                                    3.5.0
interegular                              0.3.3
jaconv                                   0.5.0
jamo                                     0.4.1
jieba                                    0.42.1
jinja2                                   3.1.6
jiter                                    0.15.0
jmespath                                 0.10.0
joblib                                   1.5.3
jsonschema                               4.26.0
jsonschema-specifications                2025.9.1
kaldiio                                  2.18.1
lark                                     1.2.2
lazy-loader                              0.5
librosa                                  0.11.0
llguidance                               1.3.0
llvmlite                                 0.44.0
lm-format-enforcer                       0.11.3
loguru                                   0.7.3
markdown-it-py                           4.2.0
markupsafe                               3.0.3
mcp                                      1.27.2
mdurl                                    0.1.2
mistral-common                           1.11.3
model-hosting-container-standards        0.1.16
modelscope                               1.37.1
mpmath                                   1.3.0
msgpack                                  1.2.0
msgspec                                  0.21.1
multidict                                6.7.1
narwhals                                 2.22.1
networkx                                 3.6.1
ninja                                    1.13.0
numba                                    0.61.2
numpy                                    2.2.6
nvidia-cublas-cu12                       12.6.4.1
nvidia-cuda-cupti-cu12                   12.6.80
nvidia-cuda-nvrtc-cu12                   12.6.77
nvidia-cuda-runtime-cu12                 12.6.77
nvidia-cudnn-cu12                        9.10.2.21
nvidia-cudnn-frontend                    1.18.0
nvidia-cufft-cu12                        11.3.0.4
nvidia-cufile-cu12                       1.11.1.6
nvidia-curand-cu12                       10.3.7.77
nvidia-cusolver-cu12                     11.7.1.2
nvidia-cusparse-cu12                     12.5.4.2
nvidia-cusparselt-cu12                   0.7.1
nvidia-cutlass-dsl                       4.5.2
nvidia-cutlass-dsl-libs-base             4.5.2
nvidia-ml-py                             13.610.43
nvidia-nccl-cu12                         2.27.5
nvidia-nvjitlink-cu12                    12.6.85
nvidia-nvshmem-cu12                      3.4.5
nvidia-nvtx-cu12                         12.6.77
omegaconf                                2.3.1
openai                                   2.41.1
openai-harmony                           0.0.8
opencv-python-headless                   4.13.0.92
opentelemetry-api                        1.42.1
opentelemetry-exporter-otlp              1.42.1
opentelemetry-exporter-otlp-proto-common 1.42.1
opentelemetry-exporter-otlp-proto-grpc   1.42.1
opentelemetry-exporter-otlp-proto-http   1.42.1
opentelemetry-proto                      1.42.1
opentelemetry-sdk                        1.42.1
opentelemetry-semantic-conventions       0.63b1
opentelemetry-semantic-conventions-ai    0.5.1
oss2                                     2.19.1
outlines-core                            0.2.11
packaging                                26.2
partial-json-parser                      0.2.1.1.post7
pillow                                   12.2.0
platformdirs                             4.10.0
pooch                                    1.9.0
prometheus-client                        0.25.0
prometheus-fastapi-instrumentator        8.0.0
propcache                                0.5.2
protobuf                                 6.33.6
psutil                                   7.2.2
py-cpuinfo                               9.0.0
pybase64                                 1.4.3
pycountry                                26.2.16
pycparser                                3.0
pycryptodome                             3.23.0
pydantic                                 2.13.4
pydantic-core                            2.46.4
pydantic-extra-types                     2.11.1
pydantic-settings                        2.14.1
pygments                                 2.20.0
pyjwt                                    2.13.0
pynndescent                              0.6.0
python-dotenv                            1.2.2
python-json-logger                       4.1.0
python-multipart                         0.0.32
pyyaml                                   6.0.3
pyzmq                                    27.1.0
quack-kernels                            0.5.0
referencing                              0.37.0
regex                                    2026.5.9
requests                                 2.34.2
rich                                     15.0.0
rich-toolkit                             0.20.1
rignore                                  0.7.6
rpds-py                                  2026.5.1
safetensors                              0.8.0
scikit-learn                             1.9.0
scipy                                    1.17.1
sentencepiece                            0.2.1
sentry-sdk                               2.62.0
setproctitle                             1.3.7
setuptools                               80.10.2
shellingham                              1.5.4
six                                      1.17.0
sniffio                                  1.3.1
soundfile                                0.14.0
soxr                                     1.1.0
sse-starlette                            3.4.4
starlette                                1.3.1
supervisor                               4.3.0
sympy                                    1.14.0
tabulate                                 0.10.0
tensorboardx                             2.6.5
threadpoolctl                            3.6.0
tiktoken                                 0.13.0
tokenizers                               0.22.2
torch                                    2.10.0+cu126
torch-c-dlpack-ext                       0.1.5
torch-complex                            0.4.4
torchaudio                               2.10.0+cu126
torchvision                              0.25.0+cu126
tqdm                                     4.68.2
transformers                             5.12.1
triton                                   3.6.0
typer                                    0.25.1
typing-extensions                        4.15.0
typing-inspection                        0.4.2
umap-learn                               0.5.12
urllib3                                  2.7.0
uvicorn                                  0.49.0
uvloop                                   0.22.1
vllm                                     0.19.1
watchfiles                               1.2.0
websockets                               16.0
xgrammar                                 0.2.2
yarl                                     1.24.2

[Chaiyym]

@LauraGPT 我尝试了你给出的版本依赖建议问题依旧出现。

1. 替换依赖版本为建议的 vLLM 0.12.0 + torch 2.9.0
2. 单独提取出了测试内容，并使用了同一个语音文件对比了AutoModel和AutoModelVLLM生成结果。
3. 排除了音频文件和模型文件可能的问题
4. AutoModel接口可以正常输出识别内容
5. AutoModelVLLM无输出识别内容

(FunASR) ➜  FunASR git:(main) ✗ uv pip list |egrep "torch|vllm"                                                        
torch                             2.9.0+cu126
torch-complex                     0.4.4
torchaudio                        2.9.0+cu126
torchvision                       0.24.0+cu126
vllm                              0.12.0
(FunASR) ➜  FunASR git:(main) ✗ VLLM_USE_V1=0 CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/test.py
funasr version: 1.3.9.
Check update of funasr, and it would cost few times. You may disable it by set `disable_update=True` in AutoModel
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
WARNING:root:trust_remote_code: True
Loading remote code successfully: model
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch
WARNING:root:trust_remote_code: False
rtf_avg: 0.016: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 11.16it/s]
rtf_avg: 0.191: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.06it/s]
rtf_avg: 0.170, time_speech:  5.547, time_escape: 0.944: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.06it/s]
[{'key': 'asr_example_zh', 'text': '欢迎大家来体验达摩院推出的语音识别模型。', 'text_tn': '欢迎大家来体验达摩院推出的语音识别模型', 'label': 'null', 'ctc_text': '欢迎大家来体验达摩院推出的语音识别模型', 'ctc_timestamps': [{'token': '欢', 'start_time': 0.42, 'end_time': 0.48, 'score': 0.999}, {'token': '迎', 'start_time': 0.6, 'end_time': 0.72, 'score': 0.993}, {'token': '大', 'start_time': 0.84, 'end_time': 0.9, 'score': 1.0}, {'token': '家', 'start_time': 0.96, 'end_time': 1.02, 'score': 1.0}, {'token': '来', 'start_time': 1.2, 'end_time': 1.26, 'score': 1.0}, {'token': '体', 'start_time': 1.5, 'end_time': 1.56, 'score': 0.998}, {'token': '验', 'start_time': 1.68, 'end_time': 1.74, 'score': 0.997}, {'token': '达', 'start_time': 1.92, 'end_time': 1.98, 'score': 0.982}, {'token': '摩', 'start_time': 2.1, 'end_time': 2.16, 'score': 0.201}, {'token': '院', 'start_time': 2.28, 'end_time': 2.34, 'score': 0.984}, {'token': '推', 'start_time': 2.52, 'end_time': 2.58, 'score': 0.997}, {'token': '出', 'start_time': 2.7, 'end_time': 2.82, 'score': 0.999}, {'token': '的', 'start_time': 2.94, 'end_time': 3.0, 'score': 1.0}, {'token': '语', 'start_time': 3.24, 'end_time': 3.3, 'score': 0.998}, {'token': '音', 'start_time': 3.42, 'end_time': 3.48, 'score': 0.995}, {'token': '识', 'start_time': 3.72, 'end_time': 3.78, 'score': 0.995}, {'token': '别', 'start_time': 3.9, 'end_time': 3.96, 'score': 0.996}, {'token': '模', 'start_time': 4.2, 'end_time': 4.26, 'score': 1.0}, {'token': '型', 'start_time': 4.38, 'end_time': 4.44, 'score': 0.998}], 'timestamps': [{'token': '欢', 'start_time': 1.03, 'end_time': 1.09, 'score': 0.999}, {'token': '迎', 'start_time': 1.21, 'end_time': 1.33, 'score': 0.993}, {'token': '大', 'start_time': 1.45, 'end_time': 1.51, 'score': 1.0}, {'token': '家', 'start_time': 1.57, 'end_time': 1.63, 'score': 1.0}, {'token': '来', 'start_time': 1.81, 'end_time': 1.87, 'score': 1.0}, {'token': '体', 'start_time': 2.11, 'end_time': 2.17, 'score': 0.998}, {'token': '验', 'start_time': 2.29, 'end_time': 2.35, 'score': 0.997}, {'token': '达', 'start_time': 2.53, 'end_time': 2.59, 'score': 0.982}, {'token': '摩', 'start_time': 2.71, 'end_time': 2.77, 'score': 0.201}, {'token': '院', 'start_time': 2.89, 'end_time': 2.95, 'score': 0.984}, {'token': '推', 'start_time': 3.13, 'end_time': 3.19, 'score': 0.997}, {'token': '出', 'start_time': 3.31, 'end_time': 3.43, 'score': 0.999}, {'token': '的', 'start_time': 3.55, 'end_time': 3.61, 'score': 1.0}, {'token': '语', 'start_time': 3.85, 'end_time': 3.91, 'score': 0.998}, {'token': '音', 'start_time': 4.03, 'end_time': 4.09, 'score': 0.995}, {'token': '识', 'start_time': 4.33, 'end_time': 4.39, 'score': 0.995}, {'token': '别', 'start_time': 4.51, 'end_time': 4.57, 'score': 0.996}, {'token': '模', 'start_time': 4.81, 'end_time': 4.87, 'score': 1.0}, {'token': '型', 'start_time': 4.99, 'end_time': 5.05, 'score': 0.998}, {'token': '。', 'start_time': 5.41, 'end_time': 5.47, 'score': 0.0}], 'timestamp': [[1030, 1090], [1210, 1330], [1450, 1510], [1570, 1630], [1810, 1870], [2110, 2170], [2290, 2350], [2530, 2590], [2710, 2770], [2890, 2950], [3130, 3190], [3310, 3430], [3550, 3610], [3850, 3910], [4030, 4090], [4330, 4390], [4510, 4570], [4810, 4870], [4990, 5050], [5410, 5470]]}]
---------------------------------------
Downloading Model from https://www.modelscope.cn to directory: /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512
INFO 06-16 16:02:40 [utils.py:253] non-default args: {'enable_prompt_embeds': True, 'trust_remote_code': True, 'dtype': 'float16', 'seed': None, 'max_model_len': 4096, 'gpu_memory_utilization': 0.8, 'disable_log_stats': True, 'enforce_eager': True, 'model': '/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
WARNING 06-16 16:02:40 [arg_utils.py:1175] `seed=None` is equivalent to `seed=0` in V1 Engine. You will no longer be allowed to pass `None` in v0.13.
INFO 06-16 16:02:40 [model.py:637] Resolved architecture: Qwen3ForCausalLM
WARNING 06-16 16:02:40 [model.py:2089] Casting torch.bfloat16 to torch.float16.
INFO 06-16 16:02:40 [model.py:1750] Using max model len 4096
INFO 06-16 16:02:40 [scheduler.py:228] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 06-16 16:02:40 [vllm.py:601] Enforce eager set, overriding optimization level to -O0
INFO 06-16 16:02:40 [vllm.py:707] Cudagraph is disabled under eager mode
WARNING 06-16 16:02:40 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:48 [core.py:93] Initializing a V1 LLM engine (v0.12.0) with config: model='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', speculative_config=None, tokenizer='/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01), seed=0, served_model_name=/home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>}, 'local_cache_dir': None}
(EngineCore_DP0 pid=35688) ERROR 06-16 16:02:48 [fa_utils.py:72] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:48 [parallel_state.py:1200] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:51157 backend=nccl
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:49 [parallel_state.py:1408] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:49 [gpu_model_runner.py:3467] Starting to load model /home/xty/.cache/modelscope/hub/models/FunAudioLLM/Fun-ASR-Nano-2512/Qwen3-0.6B-vllm...
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:50 [cuda.py:411] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION']
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.46it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.46it/s]
(EngineCore_DP0 pid=35688) 
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:50 [default_loader.py:308] Loading weights took 0.42 seconds
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:51 [gpu_model_runner.py:3549] Model loading took 1.1201 GiB memory and 0.646853 seconds
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:52 [gpu_worker.py:359] Available KV cache memory: 22.83 GiB
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:52 [kv_cache_utils.py:1286] GPU KV cache size: 213,760 tokens
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:52 [kv_cache_utils.py:1291] Maximum concurrency for 4,096 tokens per request: 52.19x
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:52 [core.py:254] init engine (profile, create kv cache, warmup model) took 1.43 seconds
(EngineCore_DP0 pid=35688) WARNING 06-16 16:02:53 [vllm.py:608] Inductor compilation was disabled by user settings,Optimizations settings that are only active duringInductor compilation will be ignored.
(EngineCore_DP0 pid=35688) INFO 06-16 16:02:53 [vllm.py:707] Cudagraph is disabled under eager mode
INFO 06-16 16:02:53 [llm.py:343] Supported tasks: ['generate']
[{'key': 'asr_example_zh', 'text': ''}]
[rank0]:[W616 16:05:06.819731435 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

[LauraGPT]

@Chaiyym 我在服务器上用完全相同的接口和模型复现了一下,AutoModelVLLM 这边是能正常出结果的:

from funasr.auto.auto_model_vllm import AutoModelVLLM
m = AutoModelVLLM(model="FunAudioLLM/Fun-ASR-Nano-2512", hub="hf",
                  tensor_parallel_size=1, gpu_memory_utilization=0.8, dtype="bf16")
res = m.generate([wav], language="中文", hotwords=["张三","北京"])
# -> [{'key':'001','text':'日本首相，啊，女首相在白宫出丑闻，还蹦迪...','timestamps':[...]}]

所以问题大概率在环境,有两个关键差异值得你排查:

1. torch CUDA build:你用的是 2.9.0+cu126,我这边能跑的是 2.9.0+cu128。
vLLM 0.12.0 的官方 wheel 是按 cu128 编的,强制配 --torch-backend=cu126 容易出现 ABI 不匹配,而 Fun-ASR-Nano 走的是 prompt-embeds 路径(自定义 embedding 输入),这种不匹配常常不报错、直接静默返回空。建议别强制 cu126,让它装默认的 cu128:

uv pip install vllm==0.12.0   # 让它自己带 torch 2.9.0+cu128

2. 你是从 vLLM 0.19.1 降到 0.12.0 的,残留缓存可能不兼容。
建议全新 venv重装,并清掉旧缓存:

rm -rf ~/.cache/vllm                      # vLLM 编译缓存
# 删掉模型目录下提取的 vLLM 权重,强制重新提取:
rm -rf ~/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/*/Qwen3-0.6B-vllm

如果还不行,麻烦贴一下:

• res 到底是 [] 还是 [{'text': ''}]?
• 完整 console 日志(尤其有没有 enable_prompt_embeds 相关的 warning / 加载 Qwen3-0.6B-vllm 那段)。

我这边可复现的完整环境:funasr 1.3.9 / vllm 0.12.0 / torch 2.9.0+cu128 / H100 / CUDA driver 550.127。先按 cu128 + 清缓存试试。

[Chaiyym]

@LauraGPT 感谢回复！很遗憾，我这边还是还是无法正常运行。
已经清空了缓存和重装环境.
做了以下尝试：

1. vllm加载Qwen3-0.6B fp16（成功）
2. AutoModel加载Fun-ASR-Nano-2512 fp16（成功）
3. AutoModelVLLM加载Fun-ASR-Nano-2512 fp16（失败）
   猜测是V100 在fp16下 AudioAdaptor 解析问题

测试脚本

import logging

from funasr import AutoModel

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    force=True
)
logger = logging.getLogger(__name__)
import sys
import torch
from funasr.auto.auto_model_vllm import AutoModelVLLM

from vllm import LLM, SamplingParams

local_file = "/data/wh/dev/FunASR/asr_example_zh.wav"

if __name__ == "__main__":
    print(f"Python 版本: {sys.version}")
    print(f"PyTorch 版本: {torch.__version__}")
    print(f"可用 GPU 数量: {torch.cuda.device_count()}")
    for i in range(torch.cuda.device_count()):
        props = torch.cuda.get_device_properties(i)
        print(
            f"GPU {i}: {props.name} | 显存: {props.total_memory / 1024 ** 3:.2f} GB | 计算能力: {props.major}.{props.minor}")

    llm = LLM(
        model='Qwen/Qwen3-0.6B',
        dtype='float16',
        trust_remote_code=True,
        gpu_memory_utilization=0.3
    )
    out = llm.generate(['<|im_start|>user\n你好<|im_end|>\n<|im_start|>assistant\n'], SamplingParams(max_tokens=50))
    print("》》》》》》》》》》LLM》》》Qwen3-0.6B 输出--------")
    print(out)
    print("-------------------------------------------")

    m = AutoModel(
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        hub="hf",
        trust_remote_code=True,
        remote_code="./model.py",
        gpu_memory_utilization=0.3,
        dtype="fp16")
    res = m.generate([local_file], language="中文", hotwords=["张三", "北京"])
    print("》》》》》》》》》》AutoModel》》》Fun-ASR-Nano-2512 输出--------")
    print(res)
    print("-------------------------------------------")

    m = AutoModelVLLM(
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        hub="hf",
        tensor_parallel_size=1,
        gpu_memory_utilization=0.3,
        dtype="fp16"
    )
    res = m.generate([local_file], language="中文", hotwords=["张三", "北京"])
    print("》》》》》》》》》》AutoModelVLLM》》》Fun-ASR-Nano-2512 输出--------")
    print(res)
    print("-------------------------------------------")

运行信息

(.vllm12) ➜  FunASR git:(main) ✗ HF_ENDPOINT=https://hf-mirror.com CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/test2.py --dtype fp16
Python 版本: 3.12.13 (main, Mar 24 2026, 22:49:22) [Clang 22.1.1 ]
PyTorch 版本: 2.9.0+cu128
可用 GPU 数量: 1
GPU 0: Tesla V100-SXM2-32GB | 显存: 31.73 GB | 计算能力: 7.0
INFO 06-17 14:53:34 [utils.py:253] non-default args: {'trust_remote_code': True, 'dtype': 'float16', 'seed': None, 'gpu_memory_utilization': 0.3, 'disable_log_stats': True}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
2026-06-17 14:53:34,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): hf-mirror.com:443
2026-06-17 14:53:35,171 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/config.json HTTP/1.1" 307 0
2026-06-17 14:53:35,427 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/config.json HTTP/1.1" 200 0
WARNING 06-17 14:53:35 [arg_utils.py:1175] `seed=None` is equivalent to `seed=0` in V1 Engine. You will no longer be allowed to pass `None` in v0.13.
2026-06-17 14:53:36,105 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
2026-06-17 14:53:36,461 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/config.json HTTP/1.1" 307 0
2026-06-17 14:53:36,663 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/config.json HTTP/1.1" 200 0
2026-06-17 14:53:37,085 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/config.json HTTP/1.1" 307 0
2026-06-17 14:53:37,201 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/config.json HTTP/1.1" 200 0
2026-06-17 14:53:37,329 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
2026-06-17 14:53:37,601 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/preprocessor_config.json HTTP/1.1" 404 0
INFO 06-17 14:53:37 [model.py:637] Resolved architecture: Qwen3ForCausalLM
WARNING 06-17 14:53:37 [model.py:2089] Casting torch.bfloat16 to torch.float16.
INFO 06-17 14:53:37 [model.py:1750] Using max model len 40960
INFO 06-17 14:53:37 [scheduler.py:228] Chunked prefill is enabled with max_num_batched_tokens=8192.
2026-06-17 14:53:37,860 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
2026-06-17 14:53:38,172 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/tokenizer_config.json HTTP/1.1" 307 0
2026-06-17 14:53:38,277 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/tokenizer_config.json HTTP/1.1" 200 0
2026-06-17 14:53:38,420 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main/additional_chat_templates?recursive=False&expand=False HTTP/1.1" 404 None
2026-06-17 14:53:38,558 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
2026-06-17 14:53:39,123 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B HTTP/1.1" 200 None
2026-06-17 14:53:39,560 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/generation_config.json HTTP/1.1" 307 0
2026-06-17 14:53:39,650 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/generation_config.json HTTP/1.1" 200 0
WARNING 06-17 14:53:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:48 [core.py:93] Initializing a V1 LLM engine (v0.12.0) with config: model='Qwen/Qwen3-0.6B', speculative_config=None, tokenizer='Qwen/Qwen3-0.6B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=40960, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01), seed=0, served_model_name=Qwen/Qwen3-0.6B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>}, 'local_cache_dir': None}
(EngineCore_DP0 pid=69852) ERROR 06-17 14:53:48 [fa_utils.py:72] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:48 [parallel_state.py:1200] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:45571 backend=nccl
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:49 [parallel_state.py:1408] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:49 [gpu_model_runner.py:3467] Starting to load model Qwen/Qwen3-0.6B...
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:50 [cuda.py:411] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION']
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:50,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): hf-mirror.com:443
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:50,959 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,083 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,314 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=False&expand=False HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,316 - filelock - DEBUG - Attempting to acquire lock 140422622673376 on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,316 - filelock - DEBUG - Lock 140422622673376 acquired on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): stats.vllm.ai:443
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,481 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/revision/main HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,614 - filelock - DEBUG - Attempting to release lock 140422622673376 on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,615 - filelock - DEBUG - Lock 140422622673376 released on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,615 - filelock - DEBUG - Attempting to acquire lock 140447644219584 on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,615 - filelock - DEBUG - Lock 140447644219584 acquired on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,860 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/model.safetensors.index.json HTTP/1.1" 404 0
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:51 [weight_utils.py:527] No model.safetensors.index.json found in remote.
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,862 - filelock - DEBUG - Attempting to release lock 140447644219584 on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:51,863 - filelock - DEBUG - Lock 140447644219584 released on /tmp/17f38bced1ab54b1066efe9c3f16bc48eb94eba3671f79537f6dc96fd508e76cQwen-Qwen3-0.6B.lock
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.10it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.10it/s]
(EngineCore_DP0 pid=69852) 
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:52 [default_loader.py:308] Loading weights took 0.50 seconds
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:52 [gpu_model_runner.py:3549] Model loading took 1.1201 GiB memory and 2.217697 seconds
(EngineCore_DP0 pid=69852) 2026-06-17 14:53:58,217 - urllib3.connectionpool - DEBUG - https://stats.vllm.ai:443 "POST / HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:58 [backends.py:655] Using cache directory: /home/xty/.cache/vllm/torch_compile_cache/d0091a5bfa/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=69852) INFO 06-17 14:53:58 [backends.py:715] Dynamo bytecode transform time: 5.73 s
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:02 [backends.py:216] Directly load the compiled graph(s) for dynamic shape from the cache, took 3.391 s
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:03 [monitor.py:34] torch.compile takes 9.12 s in total
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:04 [gpu_worker.py:359] Available KV cache memory: 6.97 GiB
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:05 [kv_cache_utils.py:1286] GPU KV cache size: 65,216 tokens
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:05 [kv_cache_utils.py:1291] Maximum concurrency for 40,960 tokens per request: 1.59x
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|█████████████████████████████████████████████████████████████████████████████| 51/51 [00:01<00:00, 29.99it/s]
Capturing CUDA graphs (decode, FULL): 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 25.16it/s]
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:09 [gpu_model_runner.py:4466] Graph capturing finished in 4 secs, took 0.45 GiB
(EngineCore_DP0 pid=69852) INFO 06-17 14:54:09 [core.py:254] init engine (profile, create kv cache, warmup model) took 16.08 seconds
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:09,971 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /Qwen/Qwen3-0.6B/resolve/main/tokenizer_config.json HTTP/1.1" 307 0
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:10,209 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "HEAD /api/resolve-cache/models/Qwen/Qwen3-0.6B/c1899de289a04d12100db370d81485cdf75e47ca/tokenizer_config.json HTTP/1.1" 200 0
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:10,316 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main/additional_chat_templates?recursive=False&expand=False HTTP/1.1" 404 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:10,423 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:10,949 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B HTTP/1.1" 200 None
(EngineCore_DP0 pid=69852) 2026-06-17 14:54:11,058 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/Qwen/Qwen3-0.6B/tree/main?recursive=True&expand=False HTTP/1.1" 200 None
INFO 06-17 14:54:11 [llm.py:343] Supported tasks: ['generate']
Adding requests: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 75.72it/s]
Processed prompts: 100%|██████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  4.17it/s, est. speed input: 37.69 toks/s, output: 209.38 toks/s]
》》》》》》》》》》LLM》》》Qwen3-0.6B 输出--------
[RequestOutput(request_id=0, prompt='<|im_start|>user\n你好<|im_end|>\n<|im_start|>assistant\n', prompt_token_ids=[151644, 872, 198, 108386, 151645, 198, 151644, 77091, 198], encoder_prompt=None, encoder_prompt_token_ids=None, prompt_logprobs=None, outputs=[CompletionOutput(index=0, text='<think>\n好的，我需要处理用户发来的消息“你好”。首先， 用户回复了“你好”，这表明这是一个友好的问候。作为AI助手，我的角色是专业而亲切的，需要按照用户的指示做出回应。\n\n在中文', token_ids=[151667, 198, 99692, 3837, 35946, 85106, 54542, 20002, 28291, 104672, 64205, 2073, 108386, 55807, 101140, 3837, 20002, 104787, 34187, 2073, 108386, 33590, 43288, 102406, 105464, 97815, 99692, 113940, 1773, 100622, 15469, 110498, 3837, 97611, 100780, 20412, 99878, 68536, 109010, 9370, 3837, 85106, 101892, 107494, 102857, 104086, 104493, 3407, 18493, 104811], cumulative_logprob=None, logprobs=None, finish_reason=length, stop_reason=None)], finished=True, metrics=None, lora_request=None, num_cached_tokens=0, multi_modal_placeholders={})]
-------------------------------------------
funasr version: 1.3.9.
Check update of funasr, and it would cost few times. You may disable it by set `disable_update=True` in AutoModel
2026-06-17 14:54:12,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): pypi.org:443
2026-06-17 14:54:14,482 - urllib3.connectionpool - DEBUG - https://pypi.org:443 "GET /pypi/funasr/json HTTP/1.1" 200 37502
You are using the latest version of funasr-1.3.9
2026-06-17 14:54:14,554 - root - INFO - download models from model hub: hf
2026-06-17 14:54:14,765 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/FunAudioLLM/Fun-ASR-Nano-2512/revision/main HTTP/1.1" 200 None
Fetching 21 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:00<00:00, 7597.72it/s]
2026-06-17 14:54:14,828 - root - WARNING - trust_remote_code: True
2026-06-17 14:54:45,058 - root - INFO - rank: 0, model is builded.
2026-06-17 14:54:45,059 - root - INFO - Loading pretrained params from /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
2026-06-17 14:54:45,067 - root - INFO - ckpt: /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
2026-06-17 14:54:46,664 - root - INFO - scope_map: ['module.', 'None']
2026-06-17 14:54:46,665 - root - INFO - excludes: None
Warning, miss key in ckpt: ctc_decoder.linear1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.linear1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.linear2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.linear2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_q.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_q.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_k.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_k.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_v.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_v.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_out.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.self_attn.linear_out.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.feed_forward.w_1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.feed_forward.w_1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.feed_forward.w_2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.feed_forward.w_2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.norm1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.norm1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.norm2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.0.norm2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_q.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_q.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_k.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_k.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_v.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_v.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_out.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.self_attn.linear_out.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.feed_forward.w_1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.feed_forward.w_1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.feed_forward.w_2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.feed_forward.w_2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.norm1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.norm1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.norm2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.1.norm2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_q.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_q.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_k.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_k.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_v.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_v.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_out.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.self_attn.linear_out.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.feed_forward.w_1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.feed_forward.w_1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.feed_forward.w_2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.feed_forward.w_2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.norm1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.norm1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.norm2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.2.norm2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_q.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_q.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_k.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_k.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_v.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_v.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_out.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.self_attn.linear_out.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.feed_forward.w_1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.feed_forward.w_1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.feed_forward.w_2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.feed_forward.w_2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.norm1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.norm1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.norm2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.3.norm2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_q.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_q.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_k.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_k.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_v.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_v.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_out.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.self_attn.linear_out.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.feed_forward.w_1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.feed_forward.w_1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.feed_forward.w_2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.feed_forward.w_2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.norm1.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.norm1.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.norm2.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc_decoder.blocks.4.norm2.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc.ctc_lo.weight, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
Warning, miss key in ckpt: ctc.ctc_lo.bias, /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
2026-06-17 14:54:47,012 - root - INFO - Loading ckpt: /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt, status: <All keys matched successfully>
rtf_avg: 0.145: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.23it/s]
》》》》》》》》》》AutoModel》》》Fun-ASR-Nano-2512 输出--------
[{'key': 'asr_example_zh', 'text': '欢迎大家来体验达摩院推出的语音识别模型。', 'text_tn': '欢迎大家来体验达摩院推出的语音识别模型', 'label': 'null', 'ctc_text': ' girl�� girl�� girl�� girlęt�� girl Rover girl�� girl�� girl�� girl�� girl��ęt�� girl��', 'ctc_timestamps': [{'token': ' girl', 'start_time': 0.0, 'end_time': 0.9, 'score': 0.0}, {'token': '��', 'start_time': 0.9, 'end_time': 0.96, 'score': 0.0}, {'token': ' girl', 'start_time': 0.96, 'end_time': 1.02, 'score': 0.0}, {'token': '��', 'start_time': 1.02, 'end_time': 1.08, 'score': 0.0}, {'token': ' girl', 'start_time': 1.08, 'end_time': 1.32, 'score': 0.0}, {'token': '��', 'start_time': 1.32, 'end_time': 1.38, 'score': 0.0}, {'token': ' girl', 'start_time': 1.38, 'end_time': 2.22, 'score': 0.0}, {'token': 'ęt', 'start_time': 2.22, 'end_time': 2.34, 'score': 0.0}, {'token': '��', 'start_time': 2.34, 'end_time': 2.4, 'score': 0.0}, {'token': ' girl', 'start_time': 2.4, 'end_time': 2.82, 'score': 0.0}, {'token': ' Rover', 'start_time': 2.82, 'end_time': 2.88, 'score': 0.0}, {'token': ' girl', 'start_time': 2.88, 'end_time': 2.94, 'score': 0.0}, {'token': '��', 'start_time': 2.94, 'end_time': 3.0, 'score': 0.0}, {'token': ' girl', 'start_time': 3.0, 'end_time': 4.08, 'score': 0.0}, {'token': '��', 'start_time': 4.08, 'end_time': 4.14, 'score': 0.0}, {'token': ' girl', 'start_time': 4.14, 'end_time': 4.5, 'score': 0.0}, {'token': '��', 'start_time': 4.5, 'end_time': 4.56, 'score': 0.0}, {'token': ' girl', 'start_time': 4.56, 'end_time': 4.8, 'score': 0.0}, {'token': '��', 'start_time': 4.8, 'end_time': 4.86, 'score': 0.0}, {'token': ' girl', 'start_time': 4.86, 'end_time': 4.92, 'score': 0.0}, {'token': '��', 'start_time': 4.92, 'end_time': 4.98, 'score': 0.0}, {'token': 'ęt', 'start_time': 4.98, 'end_time': 5.04, 'score': 0.0}, {'token': '��', 'start_time': 5.04, 'end_time': 5.1, 'score': 0.0}, {'token': ' girl', 'start_time': 5.1, 'end_time': 5.52, 'score': 0.0}, {'token': '��', 'start_time': 5.52, 'end_time': 5.58, 'score': 0.0}], 'timestamps': [{'token': '欢', 'start_time': 0.0, 'end_time': 3.06, 'score': 0.0}, {'token': '迎', 'start_time': 3.06, 'end_time': 3.12, 'score': 0.0}, {'token': '大', 'start_time': 3.12, 'end_time': 3.18, 'score': 0.0}, {'token': '家', 'start_time': 3.18, 'end_time': 4.26, 'score': 0.0}, {'token': '来', 'start_time': 4.26, 'end_time': 4.32, 'score': 0.0}, {'token': '体', 'start_time': 4.32, 'end_time': 4.56, 'score': 0.0}, {'token': '验', 'start_time': 4.56, 'end_time': 4.62, 'score': 0.0}, {'token': '达', 'start_time': 4.62, 'end_time': 4.68, 'score': 0.0}, {'token': '摩', 'start_time': 4.68, 'end_time': 4.92, 'score': 0.0}, {'token': '院', 'start_time': 4.92, 'end_time': 4.98, 'score': 0.0}, {'token': '推', 'start_time': 4.98, 'end_time': 5.04, 'score': 0.0}, {'token': '出', 'start_time': 5.04, 'end_time': 5.1, 'score': 0.0}, {'token': '的', 'start_time': 5.1, 'end_time': 5.16, 'score': 0.0}, {'token': '语', 'start_time': 5.16, 'end_time': 5.22, 'score': 0.0}, {'token': '音', 'start_time': 5.22, 'end_time': 5.28, 'score': 0.0}, {'token': '识', 'start_time': 5.28, 'end_time': 5.34, 'score': 0.0}, {'token': '别', 'start_time': 5.34, 'end_time': 5.4, 'score': 0.0}, {'token': '模', 'start_time': 5.4, 'end_time': 5.46, 'score': 0.0}, {'token': '型', 'start_time': 5.46, 'end_time': 5.52, 'score': 0.0}, {'token': '。', 'start_time': 5.52, 'end_time': 5.58, 'score': 0.0}]}]
-------------------------------------------
2026-06-17 14:54:49,940 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/FunAudioLLM/Fun-ASR-Nano-2512/revision/main HTTP/1.1" 200 None
Fetching 21 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:00<00:00, 7627.99it/s]
2026-06-17 14:54:49,985 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM model already prepared at /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm
2026-06-17 14:54:51,393 - funasr.models.fun_asr_nano.inference_vllm - INFO - Loading audio component weights from /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
2026-06-17 14:54:52,612 - funasr.models.fun_asr_nano.inference_vllm - INFO -   Loaded audio_encoder: 914 params
2026-06-17 14:54:52,618 - funasr.models.fun_asr_nano.inference_vllm - INFO -   Loaded audio_adaptor: 36 params
2026-06-17 14:54:52,993 - funasr.models.fun_asr_nano.inference_vllm - INFO - Initializing vLLM with model: /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm
2026-06-17 14:54:52,993 - funasr.models.fun_asr_nano.inference_vllm - INFO -   tensor_parallel_size=1
2026-06-17 14:54:52,993 - funasr.models.fun_asr_nano.inference_vllm - INFO -   gpu_memory_utilization=0.3
INFO 06-17 14:54:52 [utils.py:253] non-default args: {'enable_prompt_embeds': True, 'trust_remote_code': True, 'dtype': 'float16', 'seed': None, 'max_model_len': 4096, 'gpu_memory_utilization': 0.3, 'disable_log_stats': True, 'model': '/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 06-17 14:54:53 [model.py:637] Resolved architecture: Qwen3ForCausalLM
WARNING 06-17 14:54:53 [model.py:2089] Casting torch.bfloat16 to torch.float16.
INFO 06-17 14:54:53 [model.py:1750] Using max model len 4096
INFO 06-17 14:54:53 [scheduler.py:228] Chunked prefill is enabled with max_num_batched_tokens=8192.
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:02 [core.py:93] Initializing a V1 LLM engine (v0.12.0) with config: model='/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm', speculative_config=None, tokenizer='/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01), seed=0, served_model_name=/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>}, 'local_cache_dir': None}
(EngineCore_DP0 pid=70933) ERROR 06-17 14:55:02 [fa_utils.py:72] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:02 [parallel_state.py:1200] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:59575 backend=nccl
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:03 [parallel_state.py:1408] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:03 [gpu_model_runner.py:3467] Starting to load model /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm...
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:04 [cuda.py:411] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION']
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.27it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.27it/s]
(EngineCore_DP0 pid=70933) 
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:04 [default_loader.py:308] Loading weights took 0.46 seconds
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:05 [gpu_model_runner.py:3549] Model loading took 1.1201 GiB memory and 0.703009 seconds
(EngineCore_DP0 pid=70933) 2026-06-17 14:55:05,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): stats.vllm.ai:443
(EngineCore_DP0 pid=70933) 2026-06-17 14:55:07,746 - urllib3.connectionpool - DEBUG - https://stats.vllm.ai:443 "POST / HTTP/1.1" 200 None
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:11 [backends.py:655] Using cache directory: /home/xty/.cache/vllm/torch_compile_cache/a3a4fc5b93/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:11 [backends.py:715] Dynamo bytecode transform time: 5.68 s
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:15 [backends.py:216] Directly load the compiled graph(s) for dynamic shape from the cache, took 3.754 s
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:16 [monitor.py:34] torch.compile takes 9.43 s in total
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:17 [gpu_worker.py:359] Available KV cache memory: 6.97 GiB
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:17 [kv_cache_utils.py:1286] GPU KV cache size: 65,216 tokens
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:17 [kv_cache_utils.py:1291] Maximum concurrency for 4,096 tokens per request: 15.92x
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|█████████████████████████████████████████████████████████████████████████████| 51/51 [00:01<00:00, 30.36it/s]
Capturing CUDA graphs (decode, FULL): 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 25.06it/s]
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:21 [gpu_model_runner.py:4466] Graph capturing finished in 4 secs, took 0.45 GiB
(EngineCore_DP0 pid=70933) INFO 06-17 14:55:21 [core.py:254] init engine (profile, create kv cache, warmup model) took 16.44 seconds
INFO 06-17 14:55:22 [llm.py:343] Supported tasks: ['generate']
2026-06-17 14:55:24,009 - funasr.models.fun_asr_nano.inference_vllm - INFO - Loaded embedding layer: torch.Size([151936, 1024])
2026-06-17 14:55:24,097 - funasr.models.fun_asr_nano.inference_vllm - INFO - audio_tensors len: 1
2026-06-17 14:55:24,099 - funasr.models.fun_asr_nano.inference_vllm - INFO - audio_tensors data: [tensor([ 9.1553e-05,  9.1553e-05,  9.1553e-05,  ..., -1.5259e-04,
        -9.1553e-05, -6.1035e-05])]
2026-06-17 14:55:24,105 - funasr.models.fun_asr_nano.inference_vllm - INFO - speech: tensor([[[ 0.4817, -2.3440,  2.4026,  ...,  8.9150,  8.8050,  8.4805],
         [ 2.1146,  2.9222,  2.7769,  ...,  8.5283,  8.7927,  8.5231],
         [ 7.1874,  8.5743,  8.4672,  ...,  8.4838,  9.5075,  8.5775],
         ...,
         [-1.8802, -1.9553, -2.4994,  ...,  8.3596,  7.7254,  8.1835],
         [ 0.0489,  1.3407,  2.2809,  ...,  8.1754,  8.4294,  8.7627],
         [ 1.5067,  0.4971,  2.3609,  ...,  9.1858,  8.3233,  8.2384]]])
2026-06-17 14:55:24,106 - funasr.models.fun_asr_nano.inference_vllm - INFO - speech_lengths: tensor([93], dtype=torch.int32)
2026-06-17 14:55:24,190 - funasr.models.fun_asr_nano.inference_vllm - INFO - enc_out mean/std: 0.0221/0.5158
2026-06-17 14:55:24,190 - funasr.models.fun_asr_nano.inference_vllm - INFO - adp_out mean/std: 0.3896/27.3594
2026-06-17 14:55:24,191 - funasr.models.fun_asr_nano.inference_vllm - INFO - Audio encoding: 1 samples in 0.098s
2026-06-17 14:55:24,205 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM prompts: [{'prompt_embeds': tensor([[ 0.0107, -0.0374, -0.0625,  ..., -0.0051, -0.0244,  0.0014],
        [-0.0129, -0.0282, -0.0820,  ...,  0.0894,  0.0327,  0.0215],
        [ 0.0149, -0.0276, -0.0354,  ..., -0.1416,  0.0292, -0.0400],
        ...,
        [ 0.0107, -0.0374, -0.0625,  ..., -0.0051, -0.0244,  0.0014],
        [-0.0147, -0.0381, -0.0325,  ...,  0.0234, -0.0120,  0.0081],
        [ 0.0149, -0.0276, -0.0354,  ..., -0.1416,  0.0292, -0.0400]],
       device='cuda:0')}]
2026-06-17 14:55:26,436 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM generation: 2.246s
2026-06-17 14:55:26,437 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM output: [RequestOutput(request_id=0, prompt=None, prompt_token_ids=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], encoder_prompt=None, encoder_prompt_token_ids=None, prompt_logprobs=None, outputs=[CompletionOutput(index=0, text='!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!', token_ids=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], cumulative_logprob=None, logprobs=None, finish_reason=length, stop_reason=None)], finished=True, metrics=None, lora_request=None, num_cached_tokens=0, multi_modal_placeholders={})]
2026-06-17 14:55:26,438 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM output->tokenizer.decode: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
》》》》》》》》》》AutoModelVLLM》》》Fun-ASR-Nano-2512 输出--------
[{'key': 'asr_example_zh', 'text': ''}]
-------------------------------------------
[rank0]:[W617 14:55:26.787890907 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
ERROR 06-17 14:55:26 [core_client.py:600] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.
[rank0]:[W617 14:55:27.256042866 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
ERROR 06-17 14:55:27 [core_client.py:600] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.
(.vllm12) ➜  FunASR git:(main) ✗ 

[LauraGPT]

@Chaiyym 我在 GPU 服务器上用 AutoModelVLLM + FunAudioLLM/Fun-ASR-Nano-2512 把三种 dtype 都实测了一遍,定位到了问题:

dtype	结果
fp16	❌ 输出退化乱码(我这边复现为 stud stud stud... 重复;你那边表现为无输出——都是 fp16 数值不稳定导致 LLM 解码退化)
fp32	✅ 正常:日本首相啊，女首相啊。在白宫出丑闻还蹦迪…
bf16	✅ 正常

根因:Fun-ASR-Nano 的 vLLM 推理在 fp16 下数值会溢出(音频 embedding 量级较大,fp16 动态范围不够 → 退化输出)。bf16 因为指数位和 fp32 一样所以没问题——但 V100(Volta, sm_70)硬件不支持 bf16,所以你只剩 fp16(坏)和 fp32(好)两个选择,这正好解释了你「V100 fp16 下 AudioAdaptor 解析异常」的猜测,方向是对的 👍

✅ 直接可用的解决办法:V100 上请用 dtype="fp32"

model = AutoModelVLLM(
    model="FunAudioLLM/Fun-ASR-Nano-2512",
    hub="ms",
    tensor_parallel_size=1,
    gpu_memory_utilization=0.8,
    dtype="fp32",   # <-- V100 用 fp32(bf16 硬件不支持,fp16 有数值问题)
)

fp32 比 bf16 略慢、显存占用更高,但数值安全、能正常出结果,而且仍比非 vLLM 的 AutoModel 快不少。

fp16 的数值问题我们记录跟进——后续会排查能否让 fp16 路径也稳定(例如把音频 embedding 的计算/缩放固定在 fp32)。如果你在 fp32 下跑通了,麻烦回复确认一下,谢谢你这么详细的排查 🙏

[Chaiyym]

@LauraGPT 你好!
使用--dtype fp32 参数启动后还有和之前一样

1. 注意到
   torch.no_grad()调用后 我打印了一下结果adp_out 的标准差（ 27.3550）这个值似乎超出了正常值的范围
   logger.info(f"enc_out mean/std: {enc_out.mean().item():.4f}/{enc_out.std().item():.4f}")
   logger.info(f"adp_out mean/std: {adp_out.mean().item():.4f}/{adp_out.std().item():.4f}")

#2026-06-17 16:38:37,400 - funasr.models.fun_asr_nano.inference_vllm - INFO - enc_out mean/std: 0.0221/0.5155
#2026-06-17 16:38:37,401 - funasr.models.fun_asr_nano.inference_vllm - INFO - adp_out mean/std: 0.3895/27.3550

2. v100 架构如果使用的是 Tensor Core 那么只支持fp16精度

如果逻辑正确的话

• audio_encoder 写死了fp32输出
• adaptor 接收dtype参数，导致精度溢出
  问题应该出在funasr.models.fun_asr_nano.inference_vllm.FunASRNanoVLLM.generate

  adp_out, adp_lens = self.audio_adaptor(enc_out.to(dtype=self.torch_dtype), enc_lens)

3. 但是出现了矛盾点：

• 你那边H100的环境 Tensor Core 也不支持fp32 但是却可以正常运行。
• 如果使用CUDA Core 的话 H100和V100都是支持fp32的，但v100的环境还是运行失败

测试脚本

import logging

from funasr import AutoModel

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    force=True
)
logger = logging.getLogger(__name__)
import sys
import torch
from funasr.auto.auto_model_vllm import AutoModelVLLM

from vllm import LLM, SamplingParams

local_file = "/data/wh/dev/FunASR/asr_example_zh.wav"

if __name__ == "__main__":
    print(f"Python 版本: {sys.version}")
    print(f"PyTorch 版本: {torch.__version__}")
    print(f"可用 GPU 数量: {torch.cuda.device_count()}")
    for i in range(torch.cuda.device_count()):
        props = torch.cuda.get_device_properties(i)
        print(
            f"GPU {i}: {props.name} | 显存: {props.total_memory / 1024 ** 3:.2f} GB | 计算能力: {props.major}.{props.minor}")
    m = AutoModelVLLM(
        model="FunAudioLLM/Fun-ASR-Nano-2512",
        hub="hf",
        tensor_parallel_size=1,
        gpu_memory_utilization=0.9,
        dtype="fp32"
    )
    res = m.generate([local_file], language="中文", hotwords=["张三", "北京"])
    print("》》》》》》》》》》AutoModelVLLM》》》Fun-ASR-Nano-2512 输出--------")
    print(res)
    print("-------------------------------------------")

日志

(.vllm12) ➜  FunASR git:(main) ✗ HF_ENDPOINT=https://hf-mirror.com CUDA_VISIBLE_DEVICES=7 python examples/industrial_data_pretraining/fun_asr_nano/test2.py --dtype fp32   
Python 版本: 3.12.13 (main, Mar 24 2026, 22:49:22) [Clang 22.1.1 ]
PyTorch 版本: 2.9.0+cu128
可用 GPU 数量: 1
GPU 0: Tesla V100-SXM2-32GB | 显存: 31.73 GB | 计算能力: 7.0
2026-06-17 16:32:21,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): hf-mirror.com:443
2026-06-17 16:32:22,033 - urllib3.connectionpool - DEBUG - https://hf-mirror.com:443 "GET /api/models/FunAudioLLM/Fun-ASR-Nano-2512/revision/main HTTP/1.1" 200 None
Fetching 21 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:00<00:00, 94608.36it/s]
2026-06-17 16:32:22,653 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM model already prepared at /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm
2026-06-17 16:32:24,481 - funasr.models.fun_asr_nano.inference_vllm - INFO - Loading audio component weights from /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/model.pt
2026-06-17 16:32:25,797 - funasr.models.fun_asr_nano.inference_vllm - INFO -   Loaded audio_encoder: 914 params
2026-06-17 16:32:25,804 - funasr.models.fun_asr_nano.inference_vllm - INFO -   Loaded audio_adaptor: 36 params
2026-06-17 16:32:26,319 - funasr.models.fun_asr_nano.inference_vllm - INFO - Initializing vLLM with model: /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm
2026-06-17 16:32:26,319 - funasr.models.fun_asr_nano.inference_vllm - INFO -   tensor_parallel_size=1
2026-06-17 16:32:26,319 - funasr.models.fun_asr_nano.inference_vllm - INFO -   gpu_memory_utilization=0.9
INFO 06-17 16:32:26 [utils.py:253] non-default args: {'enable_prompt_embeds': True, 'trust_remote_code': True, 'seed': None, 'max_model_len': 4096, 'disable_log_stats': True, 'model': '/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
WARNING 06-17 16:32:26 [arg_utils.py:1175] `seed=None` is equivalent to `seed=0` in V1 Engine. You will no longer be allowed to pass `None` in v0.13.
INFO 06-17 16:32:26 [model.py:637] Resolved architecture: Qwen3ForCausalLM
WARNING 06-17 16:32:26 [model.py:2039] Your device 'Tesla V100-SXM2-32GB' (with compute capability 7.0) doesn't support torch.bfloat16. Falling back to torch.float16 for compatibility.
WARNING 06-17 16:32:26 [model.py:2089] Casting torch.bfloat16 to torch.float16.
INFO 06-17 16:32:26 [model.py:1750] Using max model len 4096
INFO 06-17 16:32:26 [scheduler.py:228] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 06-17 16:32:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:36 [core.py:93] Initializing a V1 LLM engine (v0.12.0) with config: model='/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm', speculative_config=None, tokenizer='/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01), seed=0, served_model_name=/home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>}, 'local_cache_dir': None}
(EngineCore_DP0 pid=37139) ERROR 06-17 16:32:36 [fa_utils.py:72] Cannot use FA version 2 is not supported due to FA2 is only supported on devices with compute capability >= 8
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:36 [parallel_state.py:1200] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.16.101.18:33779 backend=nccl
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:37 [parallel_state.py:1408] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:37 [gpu_model_runner.py:3467] Starting to load model /home/xty/.cache/huggingface/hub/models--FunAudioLLM--Fun-ASR-Nano-2512/snapshots/1a56e8599b3524387170bba98905ebfbe16c735c/Qwen3-0.6B-vllm...
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:38 [cuda.py:411] Using TRITON_ATTN attention backend out of potential backends: ['TRITON_ATTN', 'FLEX_ATTENTION']
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.78it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.78it/s]
(EngineCore_DP0 pid=37139) 
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:38 [default_loader.py:308] Loading weights took 0.38 seconds
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:39 [gpu_model_runner.py:3549] Model loading took 1.1201 GiB memory and 0.661728 seconds
(EngineCore_DP0 pid=37139) 2026-06-17 16:32:39,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): stats.vllm.ai:443
(EngineCore_DP0 pid=37139) 2026-06-17 16:32:40,884 - urllib3.connectionpool - DEBUG - https://stats.vllm.ai:443 "POST / HTTP/1.1" 200 None
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:45 [backends.py:655] Using cache directory: /home/xty/.cache/vllm/torch_compile_cache/a3a4fc5b93/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:45 [backends.py:715] Dynamo bytecode transform time: 5.91 s
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:49 [backends.py:216] Directly load the compiled graph(s) for dynamic shape from the cache, took 3.725 s
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:51 [monitor.py:34] torch.compile takes 9.63 s in total
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:52 [gpu_worker.py:359] Available KV cache memory: 26.01 GiB
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:52 [kv_cache_utils.py:1286] GPU KV cache size: 243,472 tokens
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:52 [kv_cache_utils.py:1291] Maximum concurrency for 4,096 tokens per request: 59.44x
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|█████████████████████████████████████████████████████████████████████████████| 51/51 [00:01<00:00, 30.87it/s]
Capturing CUDA graphs (decode, FULL): 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:01<00:00, 25.42it/s]
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:56 [gpu_model_runner.py:4466] Graph capturing finished in 4 secs, took 0.45 GiB
(EngineCore_DP0 pid=37139) INFO 06-17 16:32:56 [core.py:254] init engine (profile, create kv cache, warmup model) took 16.83 seconds
INFO 06-17 16:32:57 [llm.py:343] Supported tasks: ['generate']
2026-06-17 16:32:58,775 - funasr.models.fun_asr_nano.inference_vllm - INFO - Loaded embedding layer: torch.Size([151936, 1024])
2026-06-17 16:32:58,872 - funasr.models.fun_asr_nano.inference_vllm - INFO - audio_tensors len: 1
2026-06-17 16:32:58,875 - funasr.models.fun_asr_nano.inference_vllm - INFO - audio_tensors data: [tensor([ 9.1553e-05,  9.1553e-05,  9.1553e-05,  ..., -1.5259e-04,
        -9.1553e-05, -6.1035e-05])]
2026-06-17 16:32:58,886 - funasr.models.fun_asr_nano.inference_vllm - INFO - speech: tensor([[[ 0.8520, -0.6863,  2.2314,  ...,  8.7589,  8.8696,  8.6090],
         [ 1.5289,  2.3426,  2.1175,  ...,  7.9311,  8.2370,  8.2164],
         [ 7.1878,  8.5672,  8.4386,  ...,  8.1601,  7.7064,  8.3257],
         ...,
         [-1.8793, -0.2513, -1.2779,  ...,  9.0842,  7.9727,  8.2711],
         [-0.1203,  1.0908,  2.5876,  ...,  8.0178,  8.6008,  8.7574],
         [ 1.3219, -0.6706,  2.1995,  ...,  8.2719,  8.5610,  9.3694]]])
2026-06-17 16:32:58,886 - funasr.models.fun_asr_nano.inference_vllm - INFO - speech_lengths: tensor([93], dtype=torch.int32)
2026-06-17 16:32:59,089 - funasr.models.fun_asr_nano.inference_vllm - INFO - enc_out mean/std: 0.0221/0.5153
2026-06-17 16:32:59,089 - funasr.models.fun_asr_nano.inference_vllm - INFO - adp_out mean/std: 0.3879/27.3608
2026-06-17 16:32:59,090 - funasr.models.fun_asr_nano.inference_vllm - INFO - Audio encoding: 1 samples in 0.284s
2026-06-17 16:32:59,120 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM prompts: [{'prompt_embeds': tensor([[ 0.0107, -0.0374, -0.0625,  ..., -0.0051, -0.0244,  0.0014],
        [-0.0129, -0.0282, -0.0820,  ...,  0.0894,  0.0327,  0.0215],
        [ 0.0149, -0.0276, -0.0354,  ..., -0.1416,  0.0292, -0.0400],
        ...,
        [ 0.0107, -0.0374, -0.0625,  ..., -0.0051, -0.0244,  0.0014],
        [-0.0147, -0.0381, -0.0325,  ...,  0.0234, -0.0120,  0.0081],
        [ 0.0149, -0.0276, -0.0354,  ..., -0.1416,  0.0292, -0.0400]],
       device='cuda:0')}]
2026-06-17 16:33:01,354 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM generation: 2.265s
2026-06-17 16:33:01,355 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM output: [RequestOutput(request_id=0, prompt=None, prompt_token_ids=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], encoder_prompt=None, encoder_prompt_token_ids=None, prompt_logprobs=None, outputs=[CompletionOutput(index=0, text='!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!', token_ids=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], cumulative_logprob=None, logprobs=None, finish_reason=length, stop_reason=None)], finished=True, metrics=None, lora_request=None, num_cached_tokens=0, multi_modal_placeholders={})]
2026-06-17 16:33:01,355 - funasr.models.fun_asr_nano.inference_vllm - INFO - vLLM output->tokenizer.decode: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
》》》》》》》》》》AutoModelVLLM》》》Fun-ASR-Nano-2512 输出--------
[{'key': 'asr_example_zh', 'text': ''}]
-------------------------------------------
[rank0]:[W617 16:33:01.815200738 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
ERROR 06-17 16:33:01 [core_client.py:600] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.