Anyone with a few computers at home knows the problem. Together they hold plenty of memory, but no single one can load the AI model you want. Two days ago Mesh LLM hit the Hacker News front page with 339 points promising to fix that. One binary per machine, and a model too big for any single node gets cut into layer slices.
I went looking for proof and found the most honest number on the project's own benchmark page. That one number changes how I read everything else the maintainers claim.
What it is
Mesh LLM is an open-source Rust project. It gives you one OpenAI-compatible API on localhost with two modes behind it.
- Router mode sends each request to whichever machine already runs that model.
- Split mode cuts one large model into layer ranges and runs them across machines in a chain.
In split mode only tiny activations cross the network, while the heavy model weights stay on each machine.
Nobody has published a real test of it yet. So I read everything instead, the docs, the installer, the issue tracker and every thread, and checked the claims against each other. What follows is that dig, not a benchmark I ran myself.
Under the hood it's a patched llama.cpp, which the maintainers confirmed on Hacker News only when asked. The landing page still says work in progress.
What people report
The docs' path is one curl install per machine on macOS, Linux or Windows, then one command to join or host a mesh.
The evidence lives in threads. One Hacker News user joined the public mesh with a single command and wrote that it worked first try. goose, Block's agent, announced an official integration, though the post's author is also a Mesh LLM maintainer.
The maintainers' own lab numbers come from two Mac Studios on gigabit Ethernet with simulated 5 ms latency, serving GLM 5.2 with an aggressive custom quant at about 10 tokens per second.
Where it breaks
The week I researched it, the install was broken. Every official install path ends with a setup command the stable binary didn't recognize, confirmed on all three operating systems in issue #961. The advertised Homebrew tap didn't exist either. A new release landed the morning I wrote this. The issue was still open when I checked.
AMD is the weak point. An issue filed this week reports crashes on RDNA3 laptop chips, and older tickets show GPUs failing to enumerate at all. A maintainer admits they test AMD least, and CUDA carries open bugs of its own (issues #716 and #931). On the front-page thread, one user found none of the builds worked with an older GPU.
Then the physics. Their own benchmark page shows a model that fits on one Mac running at 68 tokens per second solo and 21 split across two machines.
That 21 is the number I meant.
A third machine drags it to around 13. Splitting costs three to five times the speed whenever the model fits a single box, and their roadmap admits MoE expert sharding underperforms too. The catalog claims Qwen3-235B-A22B at 16 tokens per second across two nodes, without hardware or network conditions, and Hacker News noticed within hours.
The public mesh has a harder problem. Nodes serving your request see your prompts in plaintext, and the maintainers say on the record they don't yet have a good answer for privacy or poisoned outputs. Their advice is a private mesh with trusted peers.
Verdict
This is real engineering with documentation that admits its own losses. A benchmark page that shows your product losing to a single machine takes courage.
Treat it as a tool for your own local network, not as a free cloud from strangers. If the model fits one machine, run it there. If it truly doesn't, and your machines sit on the same network, their comparison page pitches Mesh LLM as the cross-platform answer to Apple-first exo. Nothing I read contradicts that. The public mesh is a tech demo, and I wouldn't send it anything private.
The setup bug was still open when I checked, so treat the first run as a test.
curl -fsSL https://meshllm.cloud/install.sh | bash
mesh-llm serve --auto # public mesh; documented in the README at v0.73.0
- Split only what truly cannot fit one machine.
- Keep meshes inside one local network or one city. Latency, not bandwidth, is the limit.
- Apple Silicon is the tested path. CUDA has open bugs; AMD should wait a few releases.
- If a 235B model on your own hardware tempts you, watch the repo. The honest docs are a good sign.