Anyone with a few computers at home knows the problem. Together they hold plenty of memory, but no single one can load the AI model you want. Mesh LLM reached the Hacker News front page with 339 points, promising to fix that. You run one binary per machine, and a model too big for any single node gets cut into layer slices.
The most honest number sits on the project's own benchmark page, and that one number changes how you should read everything else the maintainers claim.
What it is
Mesh LLM is an open-source Rust project. It gives you one OpenAI-compatible API on localhost with two modes behind it.
- Router mode sends each request to whichever machine already runs that model.
- Split mode cuts one large model into layer ranges and runs them across machines in a chain.
In split mode only small activation values cross the network, so the large model weights stay on each machine and bandwidth is rarely the limit.
Internally, it's a patched llama.cpp, which the maintainers confirm only when asked, because the landing page still calls the project work in progress.
How you use it
The install is one curl command per machine on macOS, Linux or Windows, then one command to join or host a mesh.
You join the public mesh with a single command. goose, the open-source agent from Block, ships an official integration written by one of the Mesh LLM maintainers.
The maintainers' own lab numbers come from two Mac Studios on gigabit Ethernet with simulated 5 ms latency, serving GLM 5.2 with a heavily compressed custom quantization at about 10 tokens per second.
Where it breaks
When I set it up, the install was broken. Every official install path ends with a setup command the stable binary didn't recognize, confirmed on all three operating systems in issue #961. The advertised Homebrew tap didn't exist either. As I write this, a new release has appeared, but the setup issue is still open.
AMD is the weak point. On RDNA3 laptop chips it crashes, and older AMD GPUs fail to appear at all. A maintainer admits they test AMD least, and CUDA carries open bugs of its own (issues #716 and #931). With an older GPU, none of the builds worked.
Then the physics. Their own benchmark page shows a model that fits on one Mac running at 68 tokens per second solo and 21 split across two machines.
That 21 is the honest number I meant, because it shows the split running about three times slower than one machine.
A third machine drags it to around 13. Splitting costs three to five times the speed whenever the model already fits on a single machine, and their roadmap admits that splitting mixture-of-experts models across machines is slower still. The catalog claims Qwen3-235B-A22B at 16 tokens per second across two nodes, without listing the hardware or network conditions.
The public mesh has a harder problem, because any node that serves your request sees your prompts in plaintext, and the maintainers say on the record they have no answer yet for privacy or poisoned outputs. Their advice is a private mesh with trusted peers.
Verdict
This is real engineering, because the documentation admits its own losses. A benchmark page that shows your own product losing to a single machine takes courage.
Treat it as a tool for your own local network, not as a free cloud from strangers. If the model fits one machine, run it there. If it truly doesn't, and your machines sit on the same network, their comparison page pitches Mesh LLM as the cross-platform answer to Apple-first exo. Nothing I found contradicts that. The public mesh is a tech demo, and I wouldn't send it anything private.
As I write this, the setup bug is still open, so treat the first run as a test.
curl -fsSL https://meshllm.cloud/install.sh | bash
mesh-llm serve --auto # public mesh; README, current release v0.73.0 as I write this
- Split only what truly cannot fit one machine.
- Keep meshes inside one local network or one city. Latency, not bandwidth, is the limit.
- Apple Silicon is the tested path. CUDA has open bugs, and AMD should wait a few releases.
- If a 235B model on your own hardware tempts you, watch the repo. The honest docs are a good sign.