Contents

Underdog Saluki 27B 1.0 has been released, quantizing Qwen3.8-27B to roughly 2 bits and shrinking the model file to 7.89GB. Designed for accurate tool calling, it runs on standard llama.cpp, though it falls short of the full-precision version on some coding and math tasks.

Key Points#

  • A 27B model based on Alibaba’s Qwen3.8-27B, with its file size reduced from 54GB to 7.89GB.
  • Reportedly runs on laptops with 16GB of memory and is available on Hugging Face under the Apache 2.0 license.
  • Uses the GGUF format, compatible with standard llama.cpp and apps built on it.
  • Completed 88 of 120 tasks on Underdog Bench, surpassing the full-precision version’s 84.
  • Reported results are lower than the full-precision version on coding, math, and other tasks.

A 27B Model Reduced to 7.89GB#

Underdog Saluki 27B 1.0 quantizes Alibaba’s Qwen3.8-27B to roughly 2 bits. The standard full-precision version is 54GB, while Saluki’s file is 7.89GB. At roughly one-seventh the size, it is advertised as running on laptops with 16GB of memory.

The quantized model is based on “Qwen3.8-27B-GSQ-RCO-GGUF,” released by DASLab at the Institute of Science and Technology Austria (ISTA). Underdog modified it with a focus on tool-calling accuracy. It was released on October 8, 2026, U.S. time, and is available to download from Hugging Face.

Runs on Standard llama.cpp#

The model uses the GGUF format and works with standard llama.cpp and apps based on it. “Ternary Bonsai 2 27B,” which quantizes the same Qwen3.8-27B to ternary 2-bit values, requires a custom fork of llama.cpp, but Saluki needs no such setup.

Thinking mode is enabled by default. It can be toggled for each request, and Underdog recommends disabling it for fast tool calls. Check how to configure your app and load the model for your particular setup.

Strong Tool-Calling Performance#

Underdog’s own benchmark, “Underdog Bench,” consists of 120 tasks drawn from BFCL v4. Saluki completed 88 tasks in this test. The full-precision version completed 84, and Ternary Bonsai 2 27B completed 70.

Saluki also reportedly outperformed the full-precision version on IFEval and IFBench, which measure instruction-following ability. These results, however, come from Underdog’s own evaluations. Benchmark scores alone cannot establish accuracy for your particular use case.

Weaknesses and Caveats#

Saluki scored below the full-precision version on SWE-bench Verified, MBPP+, MuSR, and mathematical reasoning. If you prioritize bug fixing, Python coding, multistep reasoning, or math, compare performance on your actual tasks rather than choosing Saluki solely for its small size. It is also said to struggle with character-level instruction puzzles.

With thinking mode enabled, the model tends to spend longer reasoning before answering. In parallel tool calling, roughly one-fifth of responses reportedly showed minor formatting issues. If the systems you connect it to enforce strict output formats, check this before adoption.

Next Steps#

Start by checking Saluki 27B’s model files and runtime requirements on Hugging Face, then see whether your app can load it. Try a few of your intended tool calls and check both accuracy and response time before deciding to adopt it.

Source: https://pc.watch.impress.co.jp/docs/news/2147095.html

Contents