Needle 2: A 14MB Tool-Calling Model for Microcontrollers

Architecture and Memory Efficiency

Needle 2 utilizes a Simple Attention Network (SAN) architecture consisting of 27 layers and a width of 512. The model achieves its 14MB size and 28MB RAM footprint through 2-bit quantization applied to weights, activations, and the KV cache during training. To maintain a flat memory ceiling, the model employs a 256-token sliding window for attention, while system prompts and tool declarations are pinned as permanent sinks to prevent eviction. The feed-forward block is replaced by a Hadamard MLP, and 8 million of the 45 million parameters reside in an engram that requires no arithmetic to read, leaving only 35 million parameters active for matrix multiplication.

Tool Routing and Execution

Needle 2 operates exclusively as a tool-calling engine with no free-text generation path. When a request cannot be fulfilled, the model returns an empty call represented by []. For tool management, the system pins up to five tools directly into the context. If more than five tools are declared, a retrieval head embeds each schema at startup and scores the query each turn, admitting only the top five candidates. This routing ensures that unselected tools remain unreachable, keeping the memory usage constant on hardware ranging from Raspberry Pi 5s to microcontrollers like the ESP32-P4.

Performance and Limitations

Performance benchmarks vary by task. On the Seal Tools benchmark, Needle 2 leads its class, and it achieves 98.3% function name accuracy on the Google Mobile Actions set. However, the model struggles with complex multi-call requirements, scoring zero on rows requiring two consecutive calls. On the BFCL v4 benchmark, it scores 42.6 compared to 61.7 for Apple's foundation model. Users should note that all performance figures are self-reported by Cactus Compute, and the model is currently best suited for applications with a fixed, predefined set of device functions.