llm-benchmark | An open-source CLI tool to get throughputs of different models on different machines via Ollama

Several devices(computers) had been running llm-benchmark, you can see the results (tokens/sec) on Windows, macOS, and Linux. I encourage people running ollama should also run this benchmark tool to guide future people to buy next powerful machines. It’s even helpful for ollama developers to check if new changes make ollama run faster or slower.

The installation and usage is straightforward. Please visit this site to see more details.

https://llm.aidatatools.com

Installation:
pip install llm-benchmark
Usage:
llm_benchmark run

Most people used macOS to test a variety of Apple Chips, such as Apple M3 Max, Apple M2 Pro, Apple M1 Pro, or Apple M1. It gives your a flavor of how different machines generate tokens/sec on their devices via Ollama.