Fly Language Model (FLM) Wires the Full Fruit Fly Connectome Into a Frozen 1.2B LLM, and Its Own Controls Show the Wiring Does Not Help
The Fly Language Model (FLM) is a public chatbot that {couples} the full retained MaleCNS v1.0 fruit fly connectome to a frozen LiquidAI LFM2.5-1.2B-Instruct spine. The developer who created the FLM calls it the world’s first Fly Language Model, constructed on an structure known as GPF (Generative Pre-trained Fly). It doesn’t use the GPF label, explicitly disclaims being the first connectome language mannequin, and reviews that a parameter-matched management with out the fly graph performs barely higher.
Deployable: Yes, domestically. The nftechie/flm repo is MIT-licensed and runs on Python 3.12 (macOS or Linux, MPS, CUDA, or CPU) with no API key.
What was really constructed
The system is a reservoir pc bolted onto a language mannequin. All 166,700 retained nodes and 25,582,938 directed edges of the MaleCNS graph take part. The graph, the spine, and the random enter and output projections are all mounted. Only a 278,528-parameter readout is educated, which is about 0.0238% of the 1,170,340,608 spine parameters.
At every token, a mounted Gaussian projection compresses the 2,048-dimensional token embedding to 128 channels. Each reservoir node receives one channel with a random signal. The complete graph then updates with x = tanh(W(0.6x + 0.4Bc)), the place W holds incoming-normalized anatomical contact counts. States are pooled into 128 bins, handed via two educated bias-free matrices (U at 128 by 128, V at 2,048 by 128), and projected via the frozen vocabulary head as a bounded residual added to the spine logits. The residual is capped at an RMS of 0.25 throughout vocabulary coordinates.
The outcomes
On a freshly frozen set of 32 SmolTalk everyday-conversation dialogues (1,236 goal tokens), three match seeds gave:
| Condition | NLL (nats/token) |
|---|---|
| Frozen spine | 1.381995 |
| Fly readout | 1.359816 ± 0.000110 |
| Direct-input readout | 1.359328 ± 0.000108 |
| Relabeled, no refit | 1.381265 ± 0.000802 |
| No edges | 1.381995 |
The fly readout improved on the spine by 0.0222 nats per token (perplexity 3.98 to three.90). But a direct-input management, which feeds the identical 128-channel token projection straight into an an identical readout with no graph, did higher in all 3 seeds by 0.000488 nats per token. The paired bootstrap interval (+0.00000502 to +0.00104) doesn’t assist a fly-specific achieve.
Two different controls matter. Setting W to zero removes the residual precisely, reproducing the spine’s per-token losses, so the graph verifiably participates. Relabeling node identities with out retraining returns NLL close to baseline, which exhibits the readout relies on its discovered interface alignment, not that fly topology beats random wiring.
The research report additionally proves the recurrence contracts initial-state variations by at most 0.6 per token. After 10 tokens that certain is 0.00605; after 20 it’s 0.0000366. Piling in 166,700 cells doesn’t purchase lengthy reminiscence. Context nonetheless comes from the spine.
Prior work and the ‘first’ declare
The research report cites ngxson/fly-hf, an earlier prototype that used a 49,393-cell central-brain subset of MaleCNS as a reservoir educated on TinyStories with out a pretrained spine, and states plainly that it makes no declare to be the first connectome-based language mannequin. FLM’s distinction is scale (the full retained graph) and the frozen-backbone design that retains the supply of language competence identifiable.
Interactive explainer
Key Takeaways
- Full 166,700-node fly connectome drives a frozen LFM2.5-1.2B; solely 278,528 parameters prepare.
- Fly readout cuts NLL by 0.0222 nats/token, however a no-graph management beats it in each seed.
- Disconnection zeroes the residual precisely; relabeling breaks it. The graph participates, it doesn’t win.
- State forgets at 0.6 per token, so the connectome provides no long-range reminiscence.
- MIT code runs domestically on Python 3.12; research artifacts keep personal, so outcomes usually are not independently reproducible but.
Check out the Paper, GitHub repo, and live demo. Also, be at liberty to observe us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to accomplice with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so on.? Connect with us
The put up Fly Language Model (FLM) Wires the Full Fruit Fly Connectome Into a Frozen 1.2B LLM, and Its Own Controls Show the Wiring Does Not Help appeared first on MarkTechPost.
