Code Benchmarking: Precise CPU and Cache Measurements

# Benchmarking and Profiling: Accurate Code Performance Measurements

Optimizing code without objective data leads to mistakes. A rewritten hash function, supposedly to improve cache locality, slowed execution by 15%. Intuition is deceptive—precise measurements of time and processor events are needed.

A benchmarking framework solves the problem: multiple runs, statistical analysis, integration with perf. This allows identifying real bottlenecks.

High-Precision Timing Methods

The standard time() provides 1-second resolution—unacceptable for micro-optimizations.

clock_gettime()

struct timespec start, end;
clock_gettime(CLOCK_MONOTONIC, &start);
run_test();
clock_gettime(CLOCK_MONOTONIC, &end);

long ns = (end.tv_sec - start.tv_sec) * 1000000000L +
          (end.tv_nsec - start.tv_nsec);

Advantages: nanosecond resolution, resistant to system time adjustments, POSIX portability.

CPU Cycle Counters (recommended)

RISC-V:

static inline uint64_t rdcycle(void) {
    uint64_t cycles;
    asm volatile ("rdcycle %0" : "=r" (cycles));
    return cycles;
}

x86_64:

static inline uint64_t rdtsc(void) {
    uint32_t lo, hi;
    asm volatile ("rdtsc" : "=a" (lo), "=d" (hi));
    return ((uint64_t)hi << 32) | lo;
}

ARM64:

static inline uint64_t rdcycle(void) {
    uint64_t val;
    asm volatile("mrs %0, pmccntr_el0" : "=r"(val));
    return val;
}

Resolution—1 cycle, no system call overhead. Drawbacks: architecture-specific, affected by frequency changes.

Statistical Analysis of Results

A single run is useless due to cache variations, OS interrupts, branching.

Basic Statistics

#define ITERATIONS 1000
uint64_t times[ITERATIONS];
// ... fill times ...

uint64_t min = times[0], max = times[0], sum = 0;
for (int i = 0; i < ITERATIONS; i++) {
    if (times[i] < min) min = times[i];
    if (times[i] > max) max = times[i];
    sum += times[i];
}
uint64_t mean = sum / ITERATIONS;

Advanced Statistics

Median is robust to outliers, standard deviation indicates stability:

qsort(times, ITERATIONS, sizeof(uint64_t), compare_uint64);
uint64_t median = times[ITERATIONS / 2];

double variance = 0;
for (int i = 0; i < ITERATIONS; i++) {
    double diff = (double)times[i] - (double)mean;
    variance += diff * diff;
}
double stddev = sqrt(variance / ITERATIONS);

Key Metrics:

• Minimum: best case (warm cache)
• Median: typical performance
• Stddev: variability
• Maximum: worst case

Benchmarking Framework

Universal interface for reuse:

typedef struct {
    const char *name;
    void (*setup)(void);
    void (*run)(void);
    void (*teardown)(void);
} benchmark_t;

void benchmark_run(benchmark_t *bench, int iterations);

Implementation includes:

• Warm-up run
• Multiple measurements
• Statistical report

Example:

int array[1000];

void setup_array(void) {
    for (int i = 0; i < 1000; i++) {
        array[i] = i;
    }
}

void test_sequential_access(void) {
    volatile int sum = 0;
    for (int i = 0; i < 1000; i++) {
        sum += array[i];
    }
}

benchmark_t bench = {
    .name = "Sequential Array Access",
    .setup = setup_array,
    .run = test_sequential_access,
    .teardown = NULL
};

benchmark_run(&bench, 1000);

Cache Analysis with perf

perf stat tracks hardware counters:

$ perf stat -e cache-references,cache-misses ./program
  1,234,567 cache-references
     12,345 cache-misses # 1.00% miss rate

Useful Events:

• cache-references/misses: all levels
• L1-dcache-loads/misses: L1 data
• LLC-loads/misses: last level

Structure Comparison: array—1.2K misses, linked list—45K misses (37x worse).

Integrating perf into the Framework

typedef struct {
    uint64_t cycles;
    uint64_t cache_references;
    uint64_t cache_misses;
    uint64_t l1_loads;
    uint64_t l1_misses;
} perf_counters_t;

Automatic collection and aggregation of perf_event_open counters.

Common Issues and Solutions

Compiler Optimizations

// Bad: loop eliminated
int sum = 0;
for (...) sum += array[i];

// Good
volatile int sum = 0;

Cold vs Warm Cache

First run is slower. Solution: warm-up + separate metric.

Measurement Overhead

rdcycle() takes ~10 cycles. Subtract or use long tests.

System Noise

• Many iterations
• Median instead of mean
• cpupower frequency-set -g performance
• taskset -c 0
• nice -n -20

What Matters

• CPU Counters: rdtsc/rdcycle—gold standard for accuracy
• Statistics: median + stddev more reliable than mean
• perf Events: cache-misses reveal locality issues
• Cache Warm-up: essential for realistic measurements
• Volatile: prevents dead code optimization

— Editorial Team