Optimizing 1.1.1.1 DNS Cache Memory Usage

Memory Optimization Techniques

Cloudflare optimized their Big Pineapple DNS resolver by targeting heap allocation overhead and data structure padding. The following changes were implemented to reduce the per-entry memory footprint:

• Replacing Vectors with Boxes: The team replaced growable Vec types with fixed-size Box types for fields that were read-only after initial insertion. This eliminated the 8-byte capacity field and prevented overallocation of heap space for unused growth slots.
• Merging Lists and Bit-Packing: Three separate lists (answer, authority, additional) were merged into a single contiguous list. Two 16-bit unsigned integer offsets were added to delineate the sections, replacing three 16-byte headers with two 2-byte values. Boolean fields were further compressed into a single bit-flag.
• Removing Redundant Owner Names: Because the query domain is already stored as the cache key, the redundant owner name field was changed to an Option<Box<Name>>. This field is now set to None when the owner matches the query, saving a heap allocation per record.
• Boxing Large Enum Variants: To address memory padding in enums (where the size is determined by the largest variant), the team boxed large variants like TXT, SVCB, and NAPR. This allowed common, smaller variants like A and AAAA to remain inline, significantly reducing the size of the majority of cache entries.
• Storing Raw Wire Bytes: The final optimization involved storing record data as raw bytes in a single Box<[u8]> array with length-prefixed records. This eliminated per-record allocator overhead, restored cache locality, and improved performance by allowing the system to copy records directly into the outgoing DNS wire format without serialization.

Performance Results

The cumulative effect of these changes resulted in a significant reduction in memory usage and an increase in throughput. The per-entry memory footprint dropped from 953 bytes to 420 bytes, while the actual allocated memory per entry decreased from 1.1 KB to 461 bytes. Fleet-wide, this resulted in a 43% reduction in P99 resident memory usage, freeing approximately 100 terabytes of RAM. Additionally, insert throughput increased by 43% (from 625,000 to 893,000 entries per second) and lookup latency improved by 19% (from 828 nanoseconds to 670 nanoseconds).