Contents

Underdog Saluki 27B 1.0 compresses Qwen3.8-27B into a 7.89GB GGUF file, with tuning to preserve tool-calling performance. It is ready to try with stock llama.cpp, though it has weaknesses in math and parallel tool calling. Review the numbers and setup steps to decide whether it fits your needs.

Key Points#

  • The main model file is 7.89GB, smaller than the 54GB full-size version.
  • It succeeded on 88 of 120 tool-calling questions in Underdog Bench, beating the full-size version’s 84.
  • It succeeded on 42 of 100 parallel tool-calling questions, beating the full-size version’s 35.
  • Image input requires a separate 629MB or 928MB vision file.
  • It is released under the Apache 2.0 license.

Saluki’s Features#

The distributed file is Underdog-Saluki-27B-1.0-IQ2-mix.gguf. Quantization brings the 27B-parameter model down to 7.89GB. According to its Hugging Face description, it works with stock llama.cpp and apps built on it, with thinking mode enabled by default.

It succeeded on 88 of 120 tool-calling questions. The full-size Qwen3.8-27B used for comparison scored 84. These results come from the same test; they do not mean Saluki outperforms the full-size version in every use case.

Run It with llama.cpp#

Download the model with the Hugging Face CLI, then start llama-server. The following configuration sets the context length to 32,768 tokens. --jinja enables the Qwen3.8 chat template and is required for tool calling and thinking mode.

snippet.txt
huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir .
llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --jinja -ngl 99 -fa on -c 32768

The server provides an OpenAI chat API on port 8080. -ngl 99 offloads many layers to the GPU, so check whether your GPU memory and runtime environment can support it.

Use Images and Ollama#

The main model file is text-only. To handle images, download a vision file too and specify it with --mmproj. The F16 version is 928MB, and the Q8_0 version is 629MB. Once the server is running, send images through the OpenAI chat API.

snippet.txt
huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 mmproj-Underdog-Saluki-27B-1.0-F16.gguf --local-dir .
llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --mmproj mmproj-Underdog-Saluki-27B-1.0-F16.gguf --jinja -ngl 99 -fa on -c 32768

Ollama can create a model from the GGUF file. The model page reports that chat, thinking, and tool calling worked with Ollama 0.40.4 on Apple silicon.

snippet.txt
hf download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir .
echo "FROM ./Underdog-Saluki-27B-1.0-IQ2-mix.gguf" > Modelfile
ollama create saluki -f Modelfile
ollama run saluki

Performance and Caveats#

In the parallel tool-calling test, it succeeded on 42 of 100 questions, compared with 35 for the full-size version. However, around one-fifth of its parallel-calling responses reportedly contained minor formatting errors. If your tools require a strict format, check the output before integrating it into actual processing.

The gap is wider in math: Saluki scored 79.2 on AIME 2025, versus 96.7 for the full-size version. On AIME 2026, the scores were 80.0 and 94.6. The model page puts its competition math scores at roughly 82–85% of the full-size version’s. It also struggles with character-level instructions involving palindromes, vowels, and alphabetical order.

Interpret the benchmarks in the context of your intended use. Underdog Bench consists of 120 questions selected from the Berkeley Function Calling Leaderboard v4, and the page itself describes it as a small test. The published scores also include results from different evaluation environments, so testing with your own prompts and tools is the most reliable approach.

Next Step#

Start by downloading the GGUF and trying about 10 tool calls you expect to use. Once you confirm that formatting is consistent and the success rate is sufficient, add it to your actual workflow.

Source: https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0

Contents