homebench

Benchmark local LLMs for speed, memory, and quality with a live terminal leaderboard

Overview

Benchmark local LLMs for speed, memory, and quality with a live terminal leaderboard

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

From the maintainer

Origin story

I had a pile of models pulled locally and no way to answer "which of these is actually good, and what does it cost me in speed?" llama-bench gives you tok/s and nothing about output quality; lm-evaluation-harness gives you quality but isn't built around Ollama or LM Studio, which is what most people are actually running at home. So this measures both in one pass and puts them in one table.

First workflow

pip install homebench && homebench

Design choice

Local-first and honest-by-default. Every other tool does half the job: llama-bench measures speed only, lm-evaluation-harness measures quality but has no laptop UX — so homebench measures both, together, with zero config on your own machine.