Github · GitHub Repository Radar
aidatatools ollama-benchmark
LLM Benchmark for Throughput via Ollama (Local LLMs)
Stars
388
Forks
46
Watchers: 388
Language
License: MIT License
Repository Radar Score
34 / 100
Growth
- 7d
- +0
- 30d
- +0
- %
- 0.0%
Not enough metric snapshots yet to chart growth for this repository.
Score breakdown
- popularity 46
- growth 0
- activity 15
- freshness 100
- community 73
Need help integrating this stack?
Our team builds with modern open-source stacks. Tell us what you are shipping.
Get a quote →LLM Benchmark for Throughput via Ollama (Local LLMs)
Measure how fast your local LLMs really are—with a simple, cross-platform CLI tool that tells you the tokens-per-second truth.
Working Ollama installation.
python3 -m venv .venv
## On Linux and macOS
source .venv/bin/activate
## On Windows Powershell or Cmd
.\.venv\Scripts\activateuv venv .venv --python 3.13
## On Linux and macOS
source .venv/bin/activate
## On Windows Powershell or Cmd
.\.venv\Scripts\activateDepending on your python setup either
pip install llm-benchmarkor
pipx install llm-benchmarkor uv
uv pip install llm-benchmarkllm_benchmark run
It's tested on Python 3.10 and above.
7B model can be run on machines with 8GB of RAM
13B model can be run on machines with 16GB of RAM
On Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.
When memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.
ollama pull deepseek-r1:1.5b
ollama pull gemma:2b
ollama pull phi:2.7b
ollama pull phi3:3.8bWhen memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models
ollama pull phi3:3.8b
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull llama3.1:8b
ollama pull deepseek-r1:8b
ollama pull llava:7bWhen memory RAM size is greater than 15GB, but less than 31GB, it will check if these models exist. The program implicitly pull these models
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull phi4:14b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull llava:7b
ollama pull llava:13bWhen memory RAM size is greater than 31GB, it will check if these models exist. The program implicitly pull these models
ollama pull phi4:14b
ollama pull deepseek-r1:14b
ollama pull gpt-oss:20bhttps://python-poetry.org/docs/#installing-manually
python3 -m venv .venv
. ./.venv/bin/activate
pip install -U pip setuptools
pip install poetrypoetry shell
poetry install
llm_benchmark hello jasonllm_benchmark runllm_benchmark run --no-sendinfoExample #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)
llm_benchmark run --ollamabin=~/code/ollama/ollama- Create a custom benchmark file like following yaml format, replace with your own benchmark models, remember to use double quote for your model name
file_name: "custombenchmarkmodels.yml"
version: 2.0.custom
models:
- model: "deepseek-r1:1.5b"
- model: "qwen:0.5b"- run with the flag and point to the path of custombenchmarkmodels.yml
llm_benchmark run --custombenchmark=path/to/custombenchmarkmodels.yml
Repository Radar analysis
Deterministic insights derived from public metadata and our observations — not personal testing or reviews.
Why this repository is interesting
- Listed in our discovery index with public GitHub metadata for analysis.
Who should use it
- Developers working primarily with Python
- Teams exploring AI tooling, agents, or ML infrastructure
Potential use cases
- Reference or evaluate Python open-source approaches in this domain
- Prototype AI/agent workflows or study reference architectures
Strengths
- README present in our index
- Declared license: MIT License
Limitations / considerations
- Insights are derived from public metadata and our observations — not a substitute for code review
What to watch
- Radar Score is modest — dig into activity and docs before committing
Source: GitHub (public metadata) + Repository Radar analysis. We do not claim ownership of third-party repositories.
Similar repositories
browser-use/browser-use
★ 112,172 · Python · radar 67
vllm-project/vllm
★ 90,898 · Python · radar 67
huggingface/transformers
★ 164,744 · Python · radar 66
harry0703/MoneyPrinterTurbo
★ 120,187 · Python · radar 66
bytedance/deer-flow
★ 81,316 · Python · radar 66
D4Vinci/Scrapling
★ 78,252 · Python · radar 66