Eval bug: Failed to load the model

[fkpwolf]

Name and Version

$ bin/llama-cli --version
CPU_RISCV64_SPACEMIT: num_cores: 8, num_perfer_cores: 4, perfer_core_arch_id: a03c, exclude_main_thread: 0, use_ime1: 1, use_ime2: 0, mem_backend: HPAGE, cpu_mask: f, aicpu_id_offset: 0
CPU_RISCV64_SPACEMIT: alloc_chunk: open(/dev/tcm_sync_mem) failed, errno=2
CPU_RISCV64_SPACEMIT: failed to allocate init_barrier from shared mem, falling back to heap
version: 9488 (21379f9)
built with GNU 14.3.0 for Linux riscv64

Operating systems

Linux

GGML backends

CPU

Hardware

bananapif3

Models

qwen2.5-0.5b-instruct-q4_k_m.

Problem description & steps to reproduce

fan@bananapif3:~/spacemit-llama.cpp.riscv64.0.1.6$ bin/llama-cli -m ../qwen2.5-0.5b-instruct-q4_k_m.gguf -p "你好，请用一句话介绍你自己。" -n 512 -c 256

OS: Armbian 26.5.1 noble
Have installed https://github.com/spacemit-com/onnxruntime as library.

First Bad Commit

No response

Relevant log output

fan@bananapif3:~/spacemit-llama.cpp.riscv64.0.1.6$ bin/llama-cli -m ../qwen2.5-0.5b-instruct-q4_k_m.gguf -p "你好，请用一句话介绍你自己。" -n 512 -c 256
CPU_RISCV64_SPACEMIT: num_cores: 8, num_perfer_cores: 4, perfer_core_arch_id: a03c, exclude_main_thread: 0, use_ime1: 1, use_ime2: 0, mem_backend: HPAGE, cpu_mask: f, aicpu_id_offset: 0
CPU_RISCV64_SPACEMIT: alloc_chunk: open(/dev/tcm_sync_mem) failed, errno=2
CPU_RISCV64_SPACEMIT: failed to allocate init_barrier from shared mem, falling back to heap

Loading model... -0.04.971.896 E CPU_RISCV64_SPACEMIT: alloc_chunk: madvise(MADV_HUGEPAGE) failed for chunk size 536870912, errno=22
0.04.972.017 E alloc_tensor_range: failed to allocate CPU_RISCV64_SPACEMIT buffer of size 32686080
\0.05.479.637 E llama_model_load: error loading model: unable to allocate CPU_RISCV64_SPACEMIT buffer
0.05.479.886 E llama_model_load_from_file_impl: failed to load model
0.05.479.920 E common_init_from_params: failed to load model '../qwen2.5-0.5b-instruct-q4_k_m.gguf'
0.05.479.950 E srv load_model: failed to load model, '../qwen2.5-0.5b-instruct-q4_k_m.gguf'

Failed to load the model

[co-seven]

hello. I use the same version llama.cpp but i didn't reproduce your error. here is my command and it goes well. you can try again with "-t 4":
./spacemit-llama.cpp.riscv64.0.1.6/bin/llama-cli -m ./modelzoo/qwen2.5-0.5b-instruct-q4_k_m.gguf -p "你好，请用一句话介绍你自己。" -n 512 -c 256 -t 4
CPU_RISCV64_SPACEMIT: tcm is available, blk_size: 131072, blk_num: 4, is_fake_tcm: 0
CPU_RISCV64_SPACEMIT: num_cores: 8, num_perfer_cores: 4, perfer_core_arch_id: a03c, exclude_main_thread: 0, use_ime1: 1, use_ime2: 0, mem_backend: HPAGE, cpu_mask: f, aicpu_id_offset: 0
CPU_RISCV64_SPACEMIT: alloc_chunk: open(/dev/tcm_sync_mem) failed, errno=2
CPU_RISCV64_SPACEMIT: failed to allocate init_barrier from shared mem, falling back to heap

Loading model...

▄▄ ▄▄
██ ██
██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
██ ██
▀▀ ▀▀

build : b9488-21379f9d7
model : qwen2.5-0.5b-instruct-q4_k_m.gguf
modalities : text

available commands:
/exit or Ctrl+C stop or exit
/regen regenerate the last response
/clear clear the chat history
/read add a text file
/glob add text files using globbing pattern

你好，请用一句话介绍你自己。

你好，我是Qwen，由阿里云创建的人工智能助手。

[aldorof]

@fkpwolf ,
You need to enable Transparent Hugepages in your kernel:
Memory Management options -> Transparent Hugepage Support -> Transparent Hugepage Support sysfs defaults -> madvise
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
After doing it madvise(MADV_HUGEPAGE) will no longer fail.