66 lines
3.7 KiB
Markdown
66 lines
3.7 KiB
Markdown
|
---
|
||
|
|
type: xpost
|
||
|
|
source_url: https://x.com/jun_song/status/2071625328083227118
|
||
|
|
author: "@jun_song"
|
||
|
|
date: 2026-06-29
|
||
|
|
posted_by: Pit Weber (@PWeber)
|
||
|
|
topic: "Tips & Tricks (Topic 27)"
|
||
|
|
tags: [xpost, mlx, moe, local-ai, apple-silicon, mac, dense-vs-moe, deepseek, minimax, qwen, info-gap, knowledge-cutoff]
|
||
|
|
---
|
||
|
|
|
||
|
|
# Jun Song: MLX is optimized for MoE, not dense models — the local AI info gap
|
||
|
|
|
||
|
|
**Source:** [X-Post by @jun_song](https://x.com/jun_song/status/2071625328083227118)
|
||
|
|
**Posted:** 2026-06-29
|
||
|
|
**Shared by:** Pit Weber in OME-Gruppe, Topic "Tips & Tricks" (Topic 27)
|
||
|
|
|
||
|
|
## Post Content
|
||
|
|
|
||
|
|
Jun Song (@jun_song) explains the critical info gap in local AI deployment on Apple Silicon:
|
||
|
|
|
||
|
|
> Most people think Qwen3.6 27B is the best local model, so they run it on a 128GB Mac. But MLX is actually optimized for MoE (Mixture of Experts), not dense models — it fully leverages Apple's massive unified RAM. Running a 27B dense model is painfully slow on Mac. Running the dq (dequantized) version of Minimax-M3.0 or Deepseek-v4-Flash is a much smoother experience.
|
||
|
|
>
|
||
|
|
> The catch: No frontier AI can tell you this. You can't search for it, and knowledge cutoffs mean they don't have the latest facts.
|
||
|
|
>
|
||
|
|
> This info gap is why people keep saying local AI is years behind.
|
||
|
|
>
|
||
|
|
> Recommendation: Look at the latest info on experts' X accounts and follow what they do.
|
||
|
|
|
||
|
|
### Context
|
||
|
|
|
||
|
|
This post is a reply to a quoted post by @firstadopter (Tae Kim) who was complaining about running the latest Qwen on a 128GB Mac being too slow and hot, concluding that viable local LLMs are years away.
|
||
|
|
|
||
|
|
## Key Technical Points
|
||
|
|
|
||
|
|
| Point | Detail |
|
||
|
|
|-------|--------|
|
||
|
|
| **MLX optimization target** | MoE (Mixture of Experts), not dense models |
|
||
|
|
| **Apple Silicon advantage** | Massive unified RAM is ideal for MoE's sparse activation pattern |
|
||
|
|
| **Dense model penalty** | 27B dense model (e.g., Qwen3.6 27B) is "painfully slow" on Mac |
|
||
|
|
| **Recommended MoE models** | Minimax-M3.0 (dq), Deepseek-v4-Flash (dq) — dequantized versions |
|
||
|
|
| **Info gap cause** | Frontier AI models have knowledge cutoffs; can't search for cutting-edge local AI optimization info |
|
||
|
|
| **Recommendation** | Follow expert X accounts for latest local AI deployment knowledge |
|
||
|
|
|
||
|
|
## Engagement
|
||
|
|
|
||
|
|
- 954 likes
|
||
|
|
- 60 reposts
|
||
|
|
- 51 replies
|
||
|
|
- 862 bookmarks
|
||
|
|
- ~89K views
|
||
|
|
|
||
|
|
## Relevance to Our Setup
|
||
|
|
|
||
|
|
1. **Direct model overlap:** We use GLM 5.2 Cloud as primary, with **minimax-m3** and **deepseek-v4-flash** as fallbacks — the exact models Jun Song recommends for local MLX execution
|
||
|
|
2. **Validates our model choices from a different angle:** Not just cost routing (DeRonin's angle) or macro-thesis (TheProphet's angle), but **local MLX hardware optimization** — MoE models leverage Apple Silicon's unified memory architecture better than dense models
|
||
|
|
3. **Info gap thesis:** Jun Song's point that "no frontier AI can tell you this" is directly relevant to the knowledge base curation mission — this is exactly why we wikify expert knowledge from X posts
|
||
|
|
4. **Cloud-Exit connection:** Practical guide for Mac users on which models to run locally — complements the Cloud-Exit hardware thesis
|
||
|
|
5. **Dense vs. MoE:** The architectural distinction matters for deployment decisions, not just academic taxonomy
|
||
|
|
|
||
|
|
## Related Wiki Pages
|
||
|
|
|
||
|
|
- `wiki/concepts/hardware/cloud-exit-and-local-superiority.md` — Cloud-Exit thesis, local MoE on Mac
|
||
|
|
- `wiki/concepts/llm/chinese-model-cost-routing.md` — DeRonin's cost routing (same models, different angle)
|
||
|
|
- `wiki/concepts/llm/llm-model-catalog.md` — Consolidated model catalog with local deployment info
|
||
|
|
- `wiki/concepts/llm/ai-intelligence-commoditization-thesis.md` — Macro context for model accessibility
|
||
|
|
- `wiki/architecture/model-routing.md` — Our routing architecture using these models
|