What is Bench CLI?
Bench CLI is a command-line tool for systematically evaluating language model performance across structured tasks. It provides reproducible benchmarks with detailed scoring across dimensions like accuracy, coherence, instruction following, and reasoning depth.
Built for researchers and developers who need to compare model capabilities without relying on leaderboard metrics that don't reflect real-world usage patterns.
Planned Features
Usage: bench [command] [options]
Commands:
run <suite> Run a benchmark suite
compare Compare results across models
report Generate evaluation report
list List available benchmark suites
Options:
--model <id> Target model identifier
--format Output format (json, csv, markdown)
--verbose Show detailed scoring breakdown