Github trending [PrismML-Eng/Bonsai-demo] 本地化多模态模型演示

[web1992]

项目地址

https://github.com/PrismML-Eng/Bonsai-demo

AI 摘要

Bonsai Demo 是一个本地化运行 Bonsai 系列语言模型（1-bit 和 Ternary 两种格式）的演示项目。核心功能包括：支持 27B 视觉语言模型（可处理图片、截图、PDF），原生 OpenAI 风格工具调用，推理思考模式，以及 256k+ 长上下文。模型支持 Mac（Metal）、Linux/Windows（CUDA、Vulkan、ROCm）和 CPU 运行。提供快速安装脚本，一键下载模型并启动本地服务器，支持多种尺寸（27B、8B、4B、1.7B）和格式切换。

README 原文

Bonsai Demo

Website  |  GitHub  |  Discord

HF Collections: Bonsai 27B · Bonsai (1-bit) · Ternary-Bonsai

Whitepapers: Bonsai 27B · 1-bit Bonsai 8B · Ternary-Bonsai 8B

---

Using this demo repository you can run Bonsai (1-bit) and Ternary-Bonsai language models locally on Mac (Metal), Linux/Windows (CUDA, Vulkan, ROCm), or CPU.

🌱 New: Bonsai 27B

The family's newest and largest generation, and its first vision-language models (Bonsai 27B collection):

• Vision: send photos, screenshots, and PDFs; ask about them (see VISION.md).
• Agentic tool calling: native OpenAI-style tool_calls with full round-trips, plus MCP servers in both demo UIs (see TOOLS.md).
• Thinking: a reasoning model; pick the reasoning effort per chat in the UI or budget it per request.
• Long context: 256k+ token conversations.
• Tiny footprint: the 1-bit Bonsai-27B packs to ~1.125 bits per weight: it fits on a modern iPhone without memory offloading. Ternary-Bonsai-27B (~1.7 bits per weight, packed into 2-bit for fast accelerated kernels) is the higher-quality option and this demo's default.

Quick Start below gets you there in two commands: ./setup.sh downloads Ternary-Bonsai...