Multithreading in Go: CPU, caches and performance | Analysis

Multithreading in Go: How CPU, Caches, and the Scheduler Affect Performance

Modern Go applications make heavy use of parallelism, but many developers don't fully grasp what happens under the hood when working with multiple CPU cores. This article dives into the low-level mechanisms that impact multithreaded program performance: from context switching details to cache coherence. We'll explain why atomic operations and memory barriers exist, and how their quirks play out in real-world scenarios.

Multitasking Mechanisms: Cooperative vs. Preemptive

The core challenge in single-core systems is handling multiple tasks at once. Solutions came in two flavors:

• Cooperative multitasking — tasks voluntarily yield the CPU via system calls like sched_yield(). The downside is obvious: one hung task freezes the whole system (like in Windows 3.1).

• Preemptive multitasking — tasks are forcibly interrupted by a timer. Key stages:

- Generation of a hardware interrupt

- Saving the current task's context to RAM

- Selecting a new task by the scheduler

- Restoring the context

It's crucial to understand that task context includes:

• Register values (RAX, RBX, etc.)
• Instruction pointer (RIP)
• Stack pointer (RSP)
• Flags register (RFLAGS)
• Stack data

Context switching requires saving this data to RAM, which incurs overhead. On modern CPUs, a context switch takes 200–300 cycles — equivalent to executing dozens of instructions.

Multi-Core System Architecture and Cache Hierarchy

The shift to multi-core processors introduced a new issue: synchronizing access to shared memory. To understand the fixes, let's break down the cache hierarchy:

L1 Cache

• Size: 32–64 KB
• Latency: ~4 cycles
• Split into instructions (L1i) and data (L1d)
• Tied to a physical core

L2 Cache

• Size: 256 KB – 1 MB
• Latency: 10–15 cycles
• Usually tied to a core

L3 Cache

• Size: 8–40 MB
• Latency: 30–50 cycles
• Shared across all cores

Key point: the processor manages caches via a hardware controller. The OS and apps can't directly control data placement. This leads to situations where different cores work with stale data copies.

MESI Protocol and Cache Coherence

Cache coherence is handled by the MESI protocol (Modified, Exclusive, Shared, Invalid). Each cache line (typically 64 bytes) has one of four states:

• Modified: data changed in this cache and differs from RAM
• Exclusive: data matches RAM, accessible only to this core
• Shared: data matches RAM and may be in other caches
• Invalid: data is stale and can't be used

Example of write operation:

• Core A reads the variable → Shared state
• Core B reads the same variable → Shared state
• Core A writes a new value → sends invalidate request
• Core B receives the request and sets the line to Invalid

This mechanism introduces hidden delays: competing writes force cores to exchange signals over the bus, which can take 100–300 cycles.

Practical Implications for Go Developers

Grasping these mechanisms is essential when working with:

• Atomic operations: LOCK XCHG instructions ensure atomicity via bus locking
• Memory barriers: mfence controls instruction execution order
• CPU affinity: binding goroutines to specific cores via GOMAXPROCS

Example of false sharing:

var data [2]int64

// Goroutine 1
for {
    data[0]++
}

// Goroutine 2
for {
    data[1]++
}

Even though they're accessing different array elements, they land in the same cache line (64 bytes). Writing to data[0] invalidates the line for the second core, triggering constant synchronization. Fix: add padding:

var data [2]struct{
    value int64
    _     [56]byte // Padding to 64 bytes
}

Key Points

• Context switching saves state to RAM, causing ~200 cycle delays
• Cache coherence via MESI adds overhead on competing writes
• False sharing can tank performance by 2–3 orders of magnitude
• Atomic operations use bus locking or cache protocols (architecture-dependent)
• Understanding cache hierarchy lets you optimize data structures to cache line sizes

Go developers often hit these issues building high-load services. For instance, in distributed systems, poor cache handling can cause unexpected latency when processing thousands of requests per second. Key takeaway: effective parallelism demands considering not just app logic, but hardware quirks too.

For deeper dives, check the Go compiler's assembly output with the -S flag, and use profilers like perf to measure cache misses and context switches. This reveals bottlenecks invisible at the source code level.

— Editorial Team