Local AI Lab
Understand local AI
Learn what model weights, memory, hardware and inference software actually tell you. Explore sourced specifications and measured results, with the gaps made explicit.
Catalogue review recorded · individual claims have separate datesCalculationSourceMethodology
GPU database
VRAM, bandwidth, software backends and dated prices, with measurement provenance.
Apple Silicon
Each memory configuration is its own row. Unified memory is not VRAM.
PC archetypes
Five local-AI builds with target model class and limitations.
PC parts builder
Select exact parts, inspect their price sources and total your parts list.
Runtimes
llama.cpp, Ollama, LM Studio, MLX-LM, vLLM, SGLang. No invented scores.
Understand the numbers
Why model size is not the memory requirement
A model file, the working memory of an inference process, and the memory fitted to a computer are three different quantities. Comparing them directly can produce a confident but wrong answer.
Worked example: idealised 8-billion-parameter weights
8,000,000,000 × 4 bits ÷ 8 = 4,000,000,000 bytes ≈ 3.73 GiB
This is only the raw 4-bit weight payload. It is not a measured model file and does not establish that an 8B model runs in 4 GB. Real quantization formats include scales and other metadata, and some tensors can remain at higher precision.
- Weights: storage and representation
- Precision changes the weight payload. For a mixture-of-experts model, use total stored parameters here, not only the experts activated for a token. Model-card architecture and the actual file format matter.
- KV cache: context and concurrency
- Attention history can grow with tokens and simultaneous sequences. Layer count, attention design, cache precision and runtime allocation policy change its size; a context-length number alone is insufficient.
- Runtime: working space
- Temporary buffers, kernels and reserved allocations add working memory. Runtime/version and execution settings belong beside any measured result.
- Hardware: available is not installed
- Apple unified memory is shared by CPU and GPU, including other system activity. Discrete VRAM and system RAM are separate pools. Neither should be treated as completely available to one model.
Two GPUs do not automatically become one larger GPU
llama.cpp exposes different ways to split layers or tensors; they have different cache placement and communication requirements. A sum of card capacities is therefore not a universal usable-memory figure. Offloading can change where data resides without establishing useful speed.
Sources: llama.cpp multi-GPU documentation, Apple unified-memory definition. This example is arithmetic and source synthesis, not a performance test.
Inspect runtime support and its conditions →First-party measurements
0 published AIBenchIndex tests. Fixture CUDA/vLLM numbers are parser tests only and are not shown here. Methodology
No first-party measurements yet.
This lab host is a 16 GB M5 MacBook Air without llama-bench, MLX-LM, or CUDA runtimes installed, and with limited free disk. Infrastructure is in place; numbers will appear only after a real receipt passes validation.