Local vs Cloud: LLM Hardware and Inference Economics

Reading time: ≈35 minutes

Purpose: Answer one question with numbers instead of vibes: for running a large open-weight model (70B to 400B+ parameters), when does a local hardware purchase beat renting a cloud GPU or paying per token, and what actually fits on what machine.

---

本篇目录

• Table of Contents
• Data Snapshot Date
• Sizing Local Hardware with llmfit
• Benchmark Protocol Before You Buy
• Fourteen Comparable Hardware Configurations
• What Actually Fits: Named Models
• Which Local Machine for Which Usage
• Serving Engine Tuning: vLLM in Production
• Coding Agent Setup: Apple Silicon with MLX
• Cloud GPU Rental Pricing
• One-Year Cost Projections
• Power Consumption: Watts, Watt-Hours, Joules per Token
• Energy Efficiency by Model Architecture
• Cloud API Throughput: Claude vs GPT-5.6
• Why Cloud and Local Tokens/Sec Are Not Comparable
• Decision Diagram
• Decision Framework
• Sizing Self-Hosted Inference for a Team
• Switching Providers at the CLI Level