ALL INTELLIGENCE
Hacker News AIT2·Tools

Discord Ports Desktop Client to Rust to Cut Memory Use

Original title: It's All about the Allocator
58AI Score

Amid a global memory crunch driven by AI datacenter expansion, Discord is rewriting its desktop client in Rust with gpui and moving mobile apps from React Native to native iOS/Android. The piece argues that AI-assisted Rust ports alone do not fix memory bloat; allocator design and data structure choices are the real levers.

Full Report

We’re now living in a world where the global memory supply is an incredible crunch due to AI datacenter expansion. This has meant that devices with more RAM have gotten a lot more expensive. There are new devices like the MacBook Neo which have only 8 GB of RAM!

This has meant that there has been increasing scrutiny over applications that consume too much ram. Discord, Teams and Spotify are a few notorious examples since they are all just Chromium under the hood, and Chromium is bloat.

Thankfully, things are changing now. Some companies are starting to put their endless AI token budgets to good use. Discord just announced that they have started porting their whole desktop client to Rust using gpui and their mobile apps from React Native to proper native iOS/Android apps.

Just asking Claude to rewrite your stuff in Rust might not solve all of your problems though. In this Reddit post, somebody used AI to port a 12 000+ line game engine written in Typescript over to Rust. It eventually worked and passed all of the very strict tests, but was unexpectedly slow. When OP audited the code, they found that it was doing “borrow-checker appeasement” by using lots of tricks to write code that are technically memory-safe but very wasteful (cloning everywhere and using std::mem::take). The result was a program that apparently allocated several gigabytes of unnecessary memory and also spawned thousands of OS threads.

So, if you want to write truly memory-efficient code, just asking AI to do it for you might not be enough. Most code in the AI’s training data has not been written with memory-efficiency (or performance) in mind.

In order to demonstrate my point, I asked Claude for the following:

In Rust using rust-sfml, I want to draw the player’s score in the top left corner of the screen. How can I do that?

Here is what it gave me (abridged):

RUST
fn main() {
    let mut window = RenderWindow::new(
        VideoMode::new(800, 600, 32),
        "My Game",
        Style::CLOSE,
        &ContextSettings::default(),
    );
    // Load the font once, outside the game loop.
    let font = Font::from_file("assets/arial.ttf").expect("Failed to load font");

    let mut score_text = Text::new("Score: 0", &font, 24);
    score_text.set_fill_color(Color::WHITE);
    score_text.set_position((10.0, 10.0)); // small margin from the top-left

    let mut score: u32 = 0;

    while window.is_open() {
        // ... game logic that modifies `score` ...

        score_text.set_string(&format!("Score: {}", score));

        window.clear(Color::BLACK);
        // draw your game objects here
        window.draw(&score_text); // draw the HUD last so it appears on top
        window.display();
    }
}

Can you spot the issue in the main loop? That’s right, format!() returns a String and therefore it allocates a new string for the score on every frame.

What is the proper fix in this situation though? Well, it depends on how often score is updated. If it changes once a second then it would be best to update the text only when the score actually changes. If it updates closer to every frame, then the optimal solution would be to allocate one String at the start and then re-use that memory each frame:

RUST
use std::fmt::Write;

// ..

let score: i32 = 0;
let score_text_buf = String::with_capacity(32);

while window.is_open() {
  // ..

  write!(score_text_buf, "Score: {score}")
    .expect("enough space in score_text_buf");

  score_text.set_string(&score_text_buf);
  // ..
}

This is not a perfect solution as set_string() also allocates memory even if you pass it a &str because it converts it into its own string representation on the C++ side.

It might seem like a minor issue (and it is minor in this case), but you can imagine that in a project 1000x larger than this, small allocations here and there start to add up.

In Rust, there is no distinction between allocating and non-allocating functions. Every function can allocate, even something like an overloaded + operator.

Another more insidious example is this code that I wrote a while back. It was intended for parsing numbers from JSON as quickly as possible:

RUST
pub fn parse_line(line: &str) -> Coords {
    let bytes = line.as_bytes();

    let x = parse_f32(b"\"x\":", bytes, b',').expect("X found");
    let y = parse_f32(b"\"y\":", bytes, b',').expect("Y found");
    let z = parse_f32(b"\"z\":", bytes, b'}').expect("Z found");

    Coords::new(x,y,z);
}

fn parse_f32(needle: &[u8], haystack: &[u8], end_char: u8) -> Option<f32> {
    let x_start = memchr::memmem::find(haystack, needle)? + needle.len();
    let x_end = memchr(end_char, &haystack[x_start..])?;
    let x_str = unsafe { std::str::from_utf8_unchecked(&haystack[x_start..x_start + x_end]) };
    Some(x_str.parse::<f32>().expect(&format!("{} valid", x_str)))
}

Can you spot the issue now? Yes, that’s another string allocation being passed to expect(). Since we’re passing it as a parameter, that string is eagerly evaluated even when it’s not used, so most calls to parse_f32 do a completely unnecessary string allocation.

I only caught this bug when I was profiling the code and looking at the flamechart, which showed that string allocations were taking up most of the time inside parse_f32.

The fix is use a closure instead:

RUST
Some(x_str.parse::<f32>().unwrap_or_else(|_| panic!("{} Valid", x_str)))

I did ask AI to review the whole codebase to see if it could spot this, but it didn’t (there were much bigger issues with the code though).

In this situation, it would’ve been good if it was easier to see when memory is being allocated, so that this problem would’ve stood out better.

Allocations in Zig#

Zig is a language which does it differently. It is designed to be a successor to C, a systems programming language with manual memory management. In Zig, every allocation happens through the std.mem.Allocator interface. At the top level of your program, you create an allocator and then pass it around to functions that want to do allocations. Examples of allocators include std.heap.DebugAllocator, std.heap.ArenaAllocator and std.heap.FixedBufferAllocator.

The standard library and third-party libraries are built around this abstraction. This means that each memory allocation is explicit. If a function doesn’t take an allocator, then it doesn’t allocate. This is a powerful concept, as it forces you to think about when/where to allocate memory.

In Zig, there is no direct equivalent for the format!() macro in Rust. You have a couple options:

  1. If you have an instance of std.Io.Writer
ZIG
try writer.print("Score: {}", .{score});

You can get a Writer from an std.Io.File, std.Io.Writer.fixed(&buf) or std.Io.Writer.Allocating for example. 2. If you want to allocate a new slice, call allocator.print:

ZIG
const text: []u8 = try allocator.print("Score: {}", .{score});
  1. If you have a buffer, use std.mem.print:
ZIG
var buf: [32]u8 = undefined;
const slice: []u8 = try std.mem.print(&buf, "Score: {}", .{score});

For our example of a game score, we can use std.mem.printSentinel, which can add a 0-byte to the end of the string (sfml expects null-terminated strings).

ZIG
var score_text_buf: [32]u8 = undefined;

const score_text: []u8 = try std.mem.printSentinel(&score_text_buf, "Score: {}", .{score}, 0);
sf.sfText_setString(self.scoreTextShape, score_text);

Having an explicit allocator forces you to think about which approach is the most suitable. If we had more complexity in our game, we could use an ArenaAllocator to create a “scratch” for each frame to allocate from which is reset at the end of every frame.

The other canonical example for arena allocators is a web server, which might need to allocate some memory for each request. After the server has closed the connection, it can just reset the arena and serve the next request.

In programs doing any sort of heavy computations, memory access is most often the bottleneck. The CPU can perform incredible things if given the opportunity, but that doesn’t matter if it is just waiting for memory half the time. Understanding memory allocation is part of having mechanical sympathy for the machine. I think that this concept is best explained by TigerStyle from the creators of the TigerBeetle database. Though they take their ideas to the extreme by having zero dynamic memory allocations, only doing static allocations.

These ideas apply to higher-level languages too. You can speed up your Javascript code if you can understand where allocations are happening. For example, adding a number to an array with arr.push(1) vs arr = [...arr, 1]. Typed arrays can also make your number crunching code a lot faster.

Other ideas I like are Push control flow up and data flow down, Array-of-structs vs. Struct-of-arrays and Parse, don’t validate, although the last one is less about performance and more about design.