Benchmarking Concurrent Code with Multiple Parallelism Levels

Hello. Last time, you measured an HTTP service from the outside, keeping the offered request rate and client-worker limit explicit. Now you’ll measure concurrent Go code from the inside. In about 40 minutes, you’ll write a testing.B.RunParallel benchmark, run it at several goroutine counts, and distinguish its aggregate ns/op result from the latency percentiles you measured previously.

What RunParallel measures

An ordinary Go benchmark runs its operations in one benchmark goroutine. RunParallel instead starts multiple goroutines and distributes the benchmark’s total iteration count, b.N, among them. Each goroutine repeatedly calls pb.Next() until its share of the work is finished. This makes it useful for studying code that several goroutines access at once, such as a mutex-protected counter.

Start with the standard library documentation. It defines both the workload contract and a detail that is easy to misread in the output.

The distinction from the HTTP load test is important: RunParallel does not report individual-operation latency or p95. Its ns/op is the elapsed wall time of the whole parallel benchmark divided by the total completed operations. Several operations can be in progress during the same interval.

Write a benchmark with one shared resource

We’ll use a counter protected by a mutex. It is deliberately small: every operation accesses the same counter, so increasing the number of goroutines creates a meaningful shared-access workload. The mutex also preserves correctness; a fast result from a racing counter would not be a useful baseline.

In a new directory, run go mod init example.com/counterbench. Save the following as counter_bench_test.go:

package counterbench

import (
	"fmt"
	"sync"
	"testing"
)

type lockedCounter struct {
	mu sync.Mutex
	n  int64
}

func (c *lockedCounter) Add() {
	c.mu.Lock()
	c.n++
	c.mu.Unlock()
}

func BenchmarkLockedCounter(b *testing.B) {
	for _, factor := range []int{1, 2, 4, 8} {
		b.Run(fmt.Sprintf("factor=%d", factor), func(b *testing.B) {
			b.StopTimer()
			var c lockedCounter
			b.SetParallelism(factor)
			b.StartTimer()

			b.RunParallel(func(pb *testing.PB) {
				for pb.Next() {
					c.Add()
				}
			})

			b.StopTimer()
			c.mu.Lock()
			got := c.n
			c.mu.Unlock()
			if got != int64(b.N) {
				b.Fatalf("counter = %d; want %d", got, b.N)
			}
		})
	}
}

There is a new counter for each subbenchmark, shared by that subbenchmark’s worker goroutines. Each successful pb.Next() authorizes one call to Add; across all workers, those calls should total b.N. Checking the final value makes missing work visible instead of accepting a superficially fast number.

Notice where the timer calls are: before RunParallel starts its workers and after it has returned. StopTimer, StartTimer, and ResetTimer affect the benchmark globally, so do not call them inside the function passed to RunParallel. The initial setup and final check are excluded; worker startup and iteration bookkeeping remain part of the timed parallel run. A sufficiently long run makes that fixed startup cost less significant.

For a different workload, you could create scratch state at the beginning of the func(pb *testing.PB) body, outside its pb.Next() loop. Each worker would then have its own scratch state rather than racing over a shared buffer. The counter in this example is intentionally shared.

Run several levels without changing the CPU setting

Use one -cpu value for this experiment:

go test -run '^$' -bench '^BenchmarkLockedCounter$' \
  -cpu=2 -benchtime=2s -count=3 -benchmem

Here, -cpu=2 sets GOMAXPROCS to 2 for these benchmark runs. SetParallelism(factor) asks RunParallel to create the factor multiplied by GOMAXPROCS goroutines:

These are worker goroutine counts, not a promise that all workers execute simultaneously. The -2 suffix you’ll see on benchmark names denotes the GOMAXPROCS setting; it is separate from the factor in each subbenchmark name.

-benchtime=2s gives each subbenchmark time to accumulate operations, while -count=3 produces repeated results you can inspect for variability. -run '^$' skips ordinary tests for this measurement; run your correctness tests separately rather than treating a benchmark as their replacement. If the HTTP server from the previous lesson is still running, stop it before measuring to reduce competing activity.

For a race check, run a separate, shorter instrumented pass:

go test -race -run '^$' -bench '^BenchmarkLockedCounter$' \
  -cpu=2 -benchtime=100ms

Use that pass to look for races, not to compare performance with the uninstrumented results.

Interpret the sweep

For each factor, compare the repeated ns/op values, not the raw iteration counts: Go calibrates b.N, so different subbenchmarks may perform different numbers of operations. Smaller ns/op means more aggregate operations completed per unit of wall time. For example, an aggregate result of 50 ns/op corresponds to roughly 20 million completed operations per second across the benchmark, not a claim that an individual call takes 50 ns when measured alone.

You might see aggregate throughput improve, level off, or worsen as workers are added. A plateau in this counter benchmark is consistent with contention for its single mutex, but the benchmark alone does not prove how much time was spent waiting for that lock. Worker scheduling and benchmark overhead can also affect the result. Repeated measurements help you see whether a difference is stable; later profiling lessons will provide evidence about where time is going.

Keep this result beside—not in place of—the previous HTTP measurement. RunParallel answers how a selected piece of Go code behaves under concurrent access. The HTTP load test answers what requests experienced through the entire service, including effects this counter benchmark does not contain.

Takeaways

RunParallel divides one total b.N among worker goroutines. With GOMAXPROCS held fixed, SetParallelism lets you sweep goroutine counts while keeping the tested operation unchanged. Its ns/op describes aggregate wall-time throughput, not per-operation latency; check correctness, repeat runs, and avoid changing global timers inside workers.

Next, you’ll change GOMAXPROCS itself and measure what that does to a CPU-bound workload.