Hector aee635ef50 Ingest: AI Search Language Blind Spot — Daniel Nest (Why Try AI)

New raw file: raw/blog/2026-07-02_whytryai-language-blind-spot.md
New wiki page: wiki/concepts/llm/ai-search-language-blind-spot.md
  - Language Lens Bias concept + Local Sources Prompt hack
  - Denmark case study (English vs Danish search, ~0% topic overlap)
  - Global Pulse Claude Code Skill for multi-country comparisons
  - Routing parallel to chinese-model-cost-routing (model routing vs source routing)
  - Lokal vs. Global cross-section thesis expanded (5th dimension: source routing)

Cross-refs added to: mlx-moe-local-ai-optimization, local-llm-laptop-guide, chinese-model-cost-routing
Index updated (57. Update), log entry added.

2026-07-02 14:16:48 +02:00

9.9 KiB

Raw Blame History

created

updated

sources

Local LLM Laptop Guide — Best Models Without a $10k Mac Studio

TL;DR: Paul Couvert (@itsPaulAi) lists the best local models for laptop-class hardware. Top picks: Qwen3.6-27B for coding, Gemma 4 12B for everyday tasks, Parakeet 0.6B v3 for voice-to-text, Gemma 4 E4B for phone-class edge inference, and Gemma 4 26B diffusion for maximum tok/s. Key tips: use Unsloth quantized versions, run via LM Studio or llama.cpp, try Google AI Edge Gallery on mobile. Complements Jun Song's MLX MoE optimization post — both converge on the Qwen3.6-35B-A3B MoE variant.

Source

Source	Author	Date	Engagement
X-Post: "Best local models on a laptop"	Paul Couvert (@itsPaulAi)	2026-06-29	138 likes, 12 reposts, 170 bookmarks, ~13.5K views

Shared by: Pit Weber in OME-Gruppe, Topic "Tips & Tricks" (Topic 27)

The Model Stack

1. Coding: Qwen3.6-27B (or Qwen3.6-35B-A3B for speed)

Dimension	Qwen3.6-27B (Dense)	Qwen3.6-35B-A3B (MoE)
Total params	27B	35B
Active params/token	27B (all active)	3B (sparse)
Best for	Coding agents — "by far the best"	Coding, but faster
Laptop fit	Needs quantization (Unsloth)	MoE sparse activation = faster on limited hardware
Cross-ref	Jun Song warns: slow on Mac/MLX	Jun Song's MoE preference aligns

⚠️ Tension with Jun Song: Jun Song explicitly says dense 27B on Mac is "painfully slow." Paul recommends it for coding on any laptop — likely assuming Unsloth quantization on non-Mac hardware. The MoE variant (35B-A3B) is where both agree: it's the faster, better-suited option for constrained hardware. See resolution below.

2. Everyday Tasks: Gemma 4 12B

Dimension	Detail
Size	12B
Best for	Everyday tasks and questions
Why	Balanced size-to-quality ratio for general use
Runtime	LM Studio, llama.cpp, or Google AI Edge Gallery

3. Voice-to-Text: Parakeet 0.6B v3

Dimension	Detail
Size	0.6B — tiny
Best for	Speech-to-text (STT)
Claim	"Basically the best voice-to-text model you can find"
Relevance	Local Whisper alternative; relevant for OpenClaw voice interaction pipelines
Edge fit	Small enough to run on any laptop or phone

4. Edge / Phone: Gemma 4 E4B

Dimension	Detail
Size	~4B
Best for	On-device inference, offline phone use
Claim	"Surprisingly good for the size"
Mobile deployment	Google AI Edge Gallery (iOS/Android)
Cross-ref	Extends Cloud-Exit thesis to mobile — connects to OpenClaw mobile apps

5. Speed Champion: Gemma 4 26B Diffusion

Dimension	Detail
Size	26B
Type	Diffusion-based LLM
Best for	Maximum tok/s on local hardware
Claim	"The most tok/s you can have with a local model"
Architecture note	Diffusion LLMs generate tokens via iterative denoising — different paradigm from standard AR models. See post-transformer-llm-architectures.md for DeepMind's diffusion LLM research (10× efficiency gains).

Practical Deployment Tips

Quantization: Unsloth

Unsloth provides quantized model versions optimized for laptop deployment:

Benefit	Detail
Precision/size/speed	"Fantastic" balance — Paul's explicit recommendation
Compatibility	Works with LM Studio, llama.cpp
How to use	Download Unsloth quantized GGUF files → load in LM Studio or llama.cpp

Runtime Options

Runtime	Type	Best For
LM Studio	GUI app	Easy setup, model browsing, chat interface
llama.cpp	CLI / library	Maximum control, minimal overhead, embedding in apps
Google AI Edge Gallery	Mobile app	On-device inference on iOS/Android

Resolving the Dense 27B Tension

Paul recommends Qwen3.6-27B (dense) for coding. Jun Song says dense 27B on Mac is "painfully slow." This isn't a contradiction — it's a context distinction:

Factor	Paul's Context	Jun Song's Context
Hardware	Any laptop (x86, ARM, Mac)	Specifically Apple Silicon / MLX
Quantization	Unsloth quantized (explicitly recommended)	Dequantized (dq) MoE models
Framework	llama.cpp / LM Studio (CPU/GPU offload)	MLX (Apple native)
Dense 27B verdict	✅ Good with Unsloth quants	❌ "Painfully slow" on MLX
MoE 35B-A3B verdict	✅ "Faster option"	✅ Implied by MoE preference

Resolution: The performance of dense 27B depends heavily on framework + quantization:

Unsloth quantized on llama.cpp (Paul's path): tractable on commodity laptops
Unquantized/dq on MLX (Jun Song's path): slow because MLX doesn't optimize for dense
MoE variant (35B-A3B): optimal on both paths — sparse activation reduces compute per token

Recommendation: If you're on a Mac, follow Jun Song's advice (MoE models via MLX). If you're on any laptop with llama.cpp + Unsloth quants, Paul's dense 27B recommendation works. When in doubt, pick the MoE variant (Qwen3.6-35B-A3B) — it's the convergence point.

Complement to MLX MoE Optimization

This page is the cross-platform companion to mlx-moe-local-ai-optimization.md:

Dimension	MLX MoE Optimization	This Page (Laptop Guide)
Scope	Apple Silicon / MLX	Any laptop (x86, ARM, Mac)
Key insight	MoE > Dense on MLX (unified memory)	Quantized models work on any laptop
Model sources	Minimax-M3.0, DeepSeek-v4-Flash (Chinese MoE)	Qwen3.6, Gemma 4, Parakeet (broader mix)
Focus	Why MoE works on Apple Silicon	What to run + how (runtime, quantization)
Unique coverage	Info gap thesis (frontier AIs can't tell you)	STT (Parakeet), phone-edge (Gemma E4B), diffusion speed (Gemma 26B)

Together: Jun Song tells you why MoE on Mac is better. Paul tells you what to run on any laptop and how. The MoE variant Qwen3.6-35B-A3B is the overlap — both recommend it, for different reasons.

Model Landscape Summary

Use Case	Model	Size	Architecture	Hardware Floor
Coding (best)	Qwen3.6-27B	27B	Dense	Laptop w/ quantization
Coding (fast)	Qwen3.6-35B-A3B	35B/3B active	MoE	Laptop
Everyday	Gemma 4 12B	12B	Dense	Laptop
Voice-to-text	Parakeet 0.6B v3	0.6B	STT	Any device
Phone/edge	Gemma 4 E4B	~4B	Dense	Phone
Speed champion	Gemma 4 26B diffusion	26B	Diffusion LLM	Laptop (GPU helpful)

Connection to Our Setup

Model Routing Alignment

Model	Paul's Role	Our Role	Match
Qwen3.6-35B-A3B	"Faster coding option" (MoE)	Not yet in active routing — candidate	Potential addition
Qwen3.6-27B	"Best for coding agents"	Not in active routing	Reference point
Gemma 4 12B	"Everyday tasks"	Not in active routing	Consider for lightweight tasks

Cloud-Exit Extension

Paul's guide extends the Cloud-Exit thesis to laptop-class hardware — you don't need a Mac Studio, DGX Station, or Strix Halo:

Hardware Tier	Cloud-Exit Option	Source
Phone	Gemma 4 E4B via Google AI Edge Gallery	Paul Couvert
Laptop (any)	Qwen3.6/Gemma 4 with Unsloth quants via LM Studio/llama.cpp	Paul Couvert
Mac (Apple Silicon)	MoE models (Minimax-M3.0, DeepSeek-v4-Flash) via MLX	Jun Song
Prosumer desktop	AMD Strix Halo (128 GB)	../hardware/edge-inference-als-cloud-alternative.md
Enterprise desktop	NVIDIA DGX Station (748 GB)	../hardware/nvidia-dgx-station-748gb.md

Cross-References

mlx-moe-local-ai-optimization.md — Jun Song's MLX MoE post (direct complement, Apple Silicon focus)
../hardware/cloud-exit-and-local-superiority.md — Cloud-Exit thesis (this page extends to laptop tier)
../hardware/edge-inference-als-cloud-alternative.md — AMD Strix Halo (edge inference, x86)
../hardware/nvidia-dgx-station-748gb.md — NVIDIA DGX Station (enterprise tier)
llm-model-catalog.md — Consolidated model catalog
chinese-model-cost-routing.md — DeRonin's cost routing (Qwen models in cloud→local swap)
post-transformer-llm-architectures.md — Diffusion LLMs (Gemma 4 26B diffusion context)
../../tools/openclaw.md — OpenClaw mobile apps (Gemma E4B on-phone connection)
ai-search-language-blind-spot.md — "Local" als Querschnittsthema: Quellen-Routing (lokale Sprache) analog zu Edge-Inferenz (lokale Hardware)

9.9 KiB Raw Blame History Unescape Escape