DRIFT matched 130 tokens across Apple and NVIDIA GPUs. Speed was not the point
DRIFT splits one language model by layer across mismatched computers. The cross-vendor test is small, but it asks a more useful question than another tokens-per-second race: did moving the hidden state between two GPU ecosystems change the generated tokens?
A Mac using Apple's MPS backend and a Colab machine using an NVIDIA T4 generated the same 130 token IDs across three prompts while each ran a different half of Qwen2.5-1.5B-Instruct. The open-source project behind the test, DRIFT, does not present that number as a throughput win. It is a correctness check for a neutral wire between GPU ecosystems that do not normally share a distributed runtime.
The test used three prompts, a small model, and greedy decoding. The underlying logits were not identical across vendors. DRIFT's documentation says a larger logit gap could flip a later token at greater scale. The experiment shows that this split reproduced the selected token sequence for three runs. It does not establish parity across models or long contexts, but it gives people combining machines they already own a result they can try to reproduce.
A transformer cut at the layer boundary
DRIFT assigns contiguous decoder layers to separate nodes. Each node loads only its slice of the model, retains the KV cache for those layers, and receives the residual hidden state from the preceding stage. The head coordinates generation and, in chain mode, the hidden state moves directly from node to node before the final token returns.
For the main Qwen test, the model has 28 decoder layers and a hidden size of 1,536. The documented two-shard layout assigns layers 0 through 13 to one worker and 14 through 27 to the other. A decode step sends a [1, 1, 1536] fp16 hidden state plus position and token IDs. DRIFT reports 3,174 bytes per token per hop after framing. The multi-megabyte KV cache stays on the node that owns its layers.
Hugging Face's cache documentation explains that each layer stores key and value states that grow with the generated sequence. Shipping that cache across the network on every token would overwhelm DRIFT's thin boundary. The project instead reindexes each shard's kept layers to local cache slots and recomputes rotary position values from the position IDs it receives.
The project also loads weights selectively. Its published benchmark notes report 1.31 GB on each decoder shard and 0.47 GB on the orchestrator for the 3.09 GB fp16 model. The heaviest node holds 42.4% of the weights. That is the practical reason to split a model: the combined memory can hold something no single machine can.
Token equality is not numerical equality
On one Apple MPS machine, DRIFT compared a split path with a full-model reference across six prompts and 411 generated tokens. All 411 token IDs matched. The maximum reported first-step logit difference was 7.81e-03, and KL divergence was at most 2.82e-10. The documentation describes the token decisions as bitwise identical while correctly avoiding the claim that every logit was equal.
The MPS-to-CUDA test is smaller. Its 130 token IDs matched across three prompts, even though the first-step logit gap was about 2e-02, compared with about 8e-03 on one device. An argmax only changes when numerical drift alters which logit is largest. Three successful prompts cannot show how often that will happen across models, quantization settings, samplers, or long generations.
DRIFT therefore has a relaxed --prefix-match K check for cross-vendor runs. Same-device fp16 paths can be held to exact token parity. Mixed kernels may need a declared prefix requirement and a report of where divergence starts. That is a better testing habit than calling two outputs "basically the same" after reading them.
PyTorch's MPS documentation confirms that Apple GPU operations run through Metal Performance Shaders, while CUDA uses a different backend and kernel stack. DRIFT meets those runtimes at serialized bytes rather than placing both devices inside one framework-specific process group.
The benchmark refuses the obvious sales pitch
DRIFT's benchmark document opens by saying speed is the wrong axis for the project. On a single Mac, it measured 40.68 ms per token for an in-process callable and 43.13 ms for TCP plus msgpack over two localhost hops. The 2.45 ms difference was within noisy run-to-run measurements. A real LAN adds its own latency, and the repository does not publish a WAN throughput comparison.
The optional int8 wire format roughly halves each hidden-state transfer, from 3,072 bytes to 1,560 before framing. It is also lossy. The project's two-prompt check reported about 67% token match against fp16, with one prompt diverging at token 11. Int8 uses the relaxed test, not the exact one.
The project says Exo's native MLX path will usually be faster on an Apple-only cluster. DRIFT measures whether a framework-neutral split can preserve token choices while exposing its numerical drift across hardware vendors. Its evidence addresses that question, not a general performance contest.
Encryption does not prove honest computation
Without a network key, the wire is plaintext. Token IDs are reversible through the public tokenizer, so every participating node should be treated as able to read the prompt. With a shared key, DRIFT uses X25519, HKDF-SHA256, and ChaCha20-Poly1305 to encrypt and authenticate each connection.
Nodes also sign receipts that bind input and output hashes to a layer range. The head checks adjacent hashes and signatures. This can catch corruption, reordered hops, forged receipts, or a node claiming it sent different bytes. It cannot catch a node that consistently performs the wrong computation and honestly signs the bad result. DRIFT's documentation assigns that problem to recomputation audits or future redundant execution.
Failover follows a similar correctness-first design. When a node dies, the orchestrator redistributes layers, replays the sequence so far to rebuild local caches, and resumes. The repository reports 48 of 48 killed-node cases continuing with token parity. This is recovery by replay, not seamless replication.
Useful on owned machines, experimental across strangers
The operations manual supports local discovery, explicit node lists, tunnels, chain routing, and a thin head that holds no model weights. An OpenAI-shaped HTTP adapter is available too. The project's compatibility audit is candid about the gaps: DRIFT does not execute tools, guarantee strict schema-constrained decoding, or support every sampling and multimodal path.
On two computers in a trusted LAN, DRIFT can be used to study layer splitting, cache locality, transport contracts, and failure recovery. A public network of strangers needs more. The project lists Sybil resistance, leaderless consensus, economic settlement, seamless failover, and redundant execution as unfinished work.
An OpenAI-shaped endpoint can make the experiment easy to connect to api.ish.chat or another client, but compatibility at the HTTP layer does not certify the distributed computation underneath. The same point appeared in our production replay guide: test the complete system you will run.
Spare laptops have not become a datacenter. DRIFT is useful because its documentation marks the boundaries of the experiment: how many bytes cross the wire, where caches remain, which token IDs matched, how the logits differed, and which attacks the receipt scheme cannot catch. That makes the project easier to inspect and reproduce than a broad claim about decentralized inference would be.
