Memory arenas in Go: failed experiment

Hello, dear colleagues! Some time ago, I became interested in Rust and its approaches. You know, these borrow checkers that mess with your mind, lifetimes and all that stuff. Interesting, but complex to grasp. Is there something similar in Go? And I found it. A stillborn child 🥺

Introduction

Go is known for its efficient automatic Garbage Collection, GC, which greatly simplifies memory management for developers. However, for certain high-performance computing scenarios where a large number of short-lived objects are allocated and freed, GC overhead can become a bottleneck. To optimize such cases, Go engineers and community explored alternative approaches, one of which was memory arenas, introduced in Go 1.20.

What is it?

Memory arena is a memory management concept where instead of allocating and freeing each object individually, a large, pre-defined block of memory (the arena itself) is allocated. Objects needed for a specific task or during a limited time are placed sequentially inside this arena. When all objects in the arena are no longer needed, the entire arena is freed at once.

This is their main idea - instead of having GC track and collect each small object separately, we group related objects in one block and free this block entirely.

Advantages

Using arenas could potentially provide several significant benefits:

1. Reduced GC load: instead of GC scanning and processing many small objects, it only needs to manage larger arena blocks. This could reduce garbage collection pauses.
2. Faster allocation: placing objects in an arena usually comes down to a simple pointer offset within a pre-allocated block, which is a very fast operation compared to finding free space in the general heap.
3. Potential cache locality: since related objects are placed close together in memory (within one arena), this can improve processor cache locality, leading to faster data access.
4. Quick deallocation: freeing all objects in an arena happens instantly when the arena itself is freed, regardless of the number of objects inside.

Disadvantages

However, the arena concept is not without its drawbacks, especially in Go's context:

1. Manual management: you, as a developer, are responsible for choosing the right arena size and freeing the arena when it's no longer needed. This adds complexity and is a departure from the automatic GC paradigm.
2. Risk of memory leaks: if an arena is not properly freed, or if objects that should have lived only within the arena "escape" from it (i.e., references to them appear from outside the arena that outlive the arena itself), this will lead to memory leaks or incorrect program behavior.
3. Integration complexity: integrating arenas with Go's existing garbage collector and memory management model is a non-trivial task. GC must understand arenas and not attempt to collect objects managed by the arena.

Examples of Potential Use

Theoretically, arenas could be useful in the following scenarios:

• Request processing: when processing an incoming network request, allocate all temporary objects related to this request (e.g., buffers, parsing data structures) in one arena. After request processing is complete, free the entire arena.
• Data parsing: when parsing a large file or data stream, allocate data structures representing parsed elements in an arena that is freed after parsing or processing the data block is complete.
• Temporary computations: use an arena to store temporary results or intermediate data structures during complex computations that are not needed after the operation is complete.

For experiments, you can use arenas by passing the GOEXPERIMENT=arenas variable:

GOEXPERIMENT=arenas go run main.go

Let's look at a few examples:

package main
 
import (
    "fmt"
    "arena"
)
 
// Define a simple structure for example
type Data struct {
    Value int
    Label string
}
 
func main() {
    // create arena
    // arena.NewArena() creates a new memory block that will be our arena
    a := arena.NewArena()
 
    // defer arena freeing
    // using defer a.Free() ensures that the arena will be freed
    // when the main function completes its work. This simulates freeing
    // all objects in the arena at once after their lifecycle ends
    defer a.Free()
 
    // allocating objects in the arena
    // arena.New[T](a) allocates memory for one object of type T in arena 'a'
    // arena.Make[T](a, len, cap) allocates memory for a slice of type T
    // with given length and capacity in arena 'a'
 
    // allocate Data structure in the arena
    da := arena.New[Data](a)
    da.Value = 100
    da.Label = "first"
 
    // allocate Data slice in the arena
    // this will be a slice with length 5, capacity 10, memory allocated in the arena
    dataSlice := arena.MakeSlice[Data](a, 5, 10)
 
    // performing "calculations" with objects from the arena
    // fill the slice and work with objects
    sumOfValues := da.Value
    for i := range dataSlice {
        dataSlice[i].Value = i * 10
        dataSlice[i].Label = fmt.Sprintf("Data %d", i)
        sumOfValues += dataSlice[i].Value
 
        // print addresses within the arena
        fmt.Printf("object in slice [%d]: %+v (address: %p)\n", i, &dataSlice[i], &dataSlice[i])
    }
 
    // finishing work with objects from the arena
    // here logically ends the "lifecycle" of objects in the arena
    // next defer a.Free() will trigger
 
    fmt.Println("main function ending, arena will be freed")
 
    // after a.Free() triggers, the memory occupied by da and dataSlice
    // will be returned to the system/Go runtime
    // accessing these objects after freeing the arena is unsafe and can lead
    // to "use after free" errors in languages with manual memory management
    // in Go runtime that supports arenas, this means the memory is freed
    // and should not be used!
    // attempting to access values (not pointers) might temporarily work,
    // but pointers become invalid
}

Similarity with Rust Lifetimes?

Rust programmers may notice some conceptual similarity between the idea of arenas and lifetimes.

In Rust, lifetimes are a compiler mechanism that ensures references are always valid and don't point to freed memory. They allow the compiler to check how long data "lives" in memory and ensure that references to this data don't outlive the data itself. This is achieved statically, at compile time, without the need for garbage collection or reference counting.

Similarity

Both approaches (arenas in Go and lifetimes in Rust) deal with managing the lifetime of data in memory to improve performance or safety. They try to limit the "visibility" or "validity" of data to a certain scope or time period to avoid memory management issues (GC overhead or dangling references).

Key Difference

But the implementation mechanisms are fundamentally different. Lifetimes in Rust are static compiler checks that ensure memory safety directly at compile time. Arenas in Go are a dynamic allocation and deallocation strategy during program execution. Go arenas don't provide the same compile-time safety guarantees as Rust lifetimes; they are a tool for performance optimization of memory management in certain usage patterns.

Why Won't We See Arenas in Go?

Despite potential benefits, memory arenas as a publicly available, standardized feature in Go were not destined for broad production use. The main and probably insurmountable reason is backward compatibility.

Go's design and its GC are built on a fundamental assumption: any object can have references from any point in the program, and GC will handle its automatic release when it's no longer needed (i.e., when there are no more reachable references). This model is simple, safe, and forms the foundation of Go's ecosystem.

Introducing arenas as a standard feature would require significant changes in both Go runtime and standard library. Most importantly, it would violate the assumption of automatic memory management by GC for all objects. Code using arenas would require explicit arena lifecycle management, contradicting the existing model. References to objects in the arena would need to somehow differ or have restrictions to prevent GC from collecting them or to prevent the use of "dangling" references after arena release.

Integrating such a mechanism in Go without breaking existing code and maintaining language simplicity proved extremely challenging and risky. Potential backward compatibility violations and increased complexity of memory management for developers outweighed potential benefits for the general case.

While some internal or experimental projects at Google or specific libraries might use approaches similar to arenas, as a publicly available language feature in Go, memory arenas were rejected precisely due to incompatibility with the principles of backward compatibility and simplicity that Go is built upon.

Alternatives

Instead of arenas, the Go team recommends:

• Using sync.Pool for object reuse
• Optimizing algorithms and data structures
• Profiling the program to detect memory issues
• Using Buffer Pools for byte buffer management

While the idea of memory arenas seems attractive, the Go team decided that the complexity of implementation and potential issues outweigh possible benefits. This decision reflects Go's general philosophy: simplicity and reliability are more important than low-level control.

Also, I found a package that implements arenas (unofficial, of course). In certain cases, its use might make sense:

https://github.com/ortuman/nuke

Conclusions

Memory arenas are an interesting approach to optimizing memory management for specific usage patterns, offering potential reduction in GC load and faster allocation/deallocation of objects. They have some conceptual similarity with the idea of data lifetime management as implemented through lifetimes in Rust, although the mechanisms are fundamentally different.

However, for Go, with its strong emphasis on simplicity, automatic memory management, and maintaining backward compatibility, integrating arenas as a standardized feature proved too complex and violating of foundational principles. Thus, despite their potential benefits in certain scenarios, arenas were not destined to become part of core Go, and developers continue to rely on improvements to the built-in garbage collector for performance optimization.