gpt-oss-20b on a 64 GB Mac: speed, quality, memory
OpenAI, released Aug 2025. 21B parameters, 3.6B active (mixture of experts). MXFP4 MLX build, 12.1 GB download. License Apache-2.0.
- HumanEval+ pass@1: 143 of 164 (87.2%), rank 4 of 8
- HumanEval pass@1: 152 of 164 (92.7%)
- Generation speed: 82.6 tokens per second (5 trials, 82.4 to 83.0); prompt 1,782 tokens per second
- Peak memory: 12.7 GB · power while generating: 22 W · GPU peak 70 °C
Run it yourself
pip install mlx-lm mlx_lm.chat --model mlx-community/gpt-oss-20b-MXFP4-Q8
HumanEval/16
Count the distinct characters in a string, ignoring upper and lower case.
It only counted letters. The spaces were left out, so 'Jerry jERRY JeRRRY' gave 4, not 5.
Problems it failed
HumanEval/16, HumanEval/39, HumanEval/57, HumanEval/76, HumanEval/103, HumanEval/121, HumanEval/127, HumanEval/134, HumanEval/145, HumanEval/147, HumanEval/149, HumanEval/151
Raw files: speed.log, evaluate.log, eval_results.json