Go From Zero, Part 8: sync, context, and Concurrency Patterns

This is Part 8 of 12 in Go From Zero, a beginner-to-production series on the Go programming language. If you missed it, Part 7: Goroutines, Channels, and select is the place to catch up.

In Part 7 we met goroutines, channels and select. Those are the raw ingredients. Today we cook with them. You’ll learn how to protect shared data, how to stop work that’s no longer needed, and the handful of patterns that show up again and again in real Go services.

When channels aren’t the right tool: sync.Mutex

Channels are great for passing data along. But sometimes you just have a piece of shared state, like a counter or a cache, that several goroutines need to update. For that, a mutex (mutual exclusion lock) is simpler.

Here’s the bug first. Run it with go run -race . and the race detector will shout at you:

package main

import (
    "fmt"
    "sync"
)

func main() {
    count := 0
    var wg sync.WaitGroup
    for range 1000 {
        wg.Add(1)
        go func() {
            defer wg.Done()
            count++ // DATA RACE: many goroutines write at once
        }()
    }
    wg.Wait()
    fmt.Println(count) // often less than 1000!
}

count++ looks like one step but is actually three: read, add, write. Two goroutines can read the same old value and both write back the same new one, losing an update.

The fix is to guard the shared data with a mutex:

package main

import (
    "fmt"
    "sync"
)

// SafeCounter is safe to use from many goroutines.
type SafeCounter struct {
    mu     sync.Mutex
    counts map[string]int
}

func NewSafeCounter() *SafeCounter {
    return &SafeCounter{counts: make(map[string]int)}
}

func (c *SafeCounter) Inc(key string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.counts[key]++
}

func (c *SafeCounter) Value(key string) int {
    c.mu.Lock()
    defer c.mu.Unlock()
    return c.counts[key]
}

func main() {
    c := NewSafeCounter()
    var wg sync.WaitGroup
    for range 1000 {
        wg.Add(1)
        go func() {
            defer wg.Done()
            c.Inc("visits")
        }()
    }
    wg.Wait()
    fmt.Println(c.Value("visits")) // always 1000
}

A few habits shown here:

• Keep the mutex next to the data it protects, inside the same struct, and keep both unexported.
• Lock then immediately defer Unlock so you can’t forget to unlock, even if the function returns early or panics.
• Use pointer receivers. Copying a struct copies its mutex, and a copied mutex protects nothing. go vet warns you about this.

RWMutex for read-heavy data

If many goroutines read and only a few write, sync.RWMutex lets readers proceed in parallel. Use RLock/RUnlock for reads and Lock/Unlock for writes. Don’t reach for it by default though. A plain Mutex is simpler and often just as fast.

Other useful pieces of the sync package

• sync.Once runs something exactly once, even if called from many goroutines. Handy for lazy initialisation.
• sync/atomic provides lock-free operations like atomic.Int64 for simple counters.
• golang.org/x/sync/errgroup (an official extended package) is a WaitGroup that also collects the first error and cancels the rest. You’ll love it in real services.

How to choose: channel or mutex?

A practical rule:

• Passing ownership of data, distributing work, or signalling events? Use a channel.
• Protecting shared state, like a cache or counter? Use a mutex.

Both are idiomatic. Use whichever makes the code easier to read.

context: cancelling work that’s no longer needed

Imagine a user opens a page, your server starts three database queries and an API call, and then the user closes the tab. Should all that work keep running? Of course not. Go’s answer is the context package.

A context.Context carries a cancellation signal and an optional deadline through your call chain. Every function that does I/O or might take a while should accept one as its first argument, named ctx.

package main

import (
    "context"
    "errors"
    "fmt"
    "time"
)

// slowQuery pretends to be a database call that takes `d` to complete.
func slowQuery(ctx context.Context, d time.Duration) (string, error) {
    select {
    case <-time.After(d):
        return "query result", nil
    case <-ctx.Done(): // closed when the context is cancelled or times out
        return "", ctx.Err()
    }
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
    defer cancel() // always call cancel to release resources

    res, err := slowQuery(ctx, 50*time.Millisecond)
    fmt.Println(res, err) // query result <nil>

    res, err = slowQuery(ctx, 500*time.Millisecond)
    fmt.Println(res, err)                                 // context deadline exceeded
    fmt.Println(errors.Is(err, context.DeadlineExceeded)) // true
}

The key line is case <-ctx.Done(). Done() returns a channel that gets closed when the context is cancelled, so any goroutine can select on it and stop early.

Contexts form a tree. When a parent is cancelled, all its children are cancelled too. In an HTTP server, every request already comes with a context (r.Context()) that’s cancelled when the client disconnects. Pass it down, and everything stops cleanly.

The ways to create one:

• context.Background(): the empty root, used in main and tests.
• context.WithCancel(parent): gives you a cancel function to call yourself.
• context.WithTimeout(parent, d) / WithDeadline: cancels automatically after a duration or at a time.
• context.WithValue(parent, key, val): carries request-scoped values like a request ID. Use sparingly, never for optional function parameters.

Context rules worth following

• ctx is the first parameter: func Fetch(ctx context.Context, id string).
• Don’t store contexts in structs. Pass them explicitly through each call.
• Always defer cancel() right after creating a cancellable context.
• Never pass a nil context. Use context.TODO() if you’re not sure yet.

Pattern 1: worker pool

You have 100 jobs and want at most 5 running at once, maybe to avoid overwhelming a database or a rate-limited API. That’s a worker pool.

package main

import (
    "fmt"
    "sync"
    "time"
)

func worker(id int, jobs <-chan int, results chan<- string) {
    for j := range jobs {
        time.Sleep(20 * time.Millisecond) // simulate work
        results <- fmt.Sprintf("worker %d finished job %d", id, j)
    }
}

func main() {
    const numJobs = 10
    const numWorkers = 3

    jobs := make(chan int)
    results := make(chan string)

    var wg sync.WaitGroup
    for w := 1; w <= numWorkers; w++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            worker(w, jobs, results)
        }()
    }

    // send jobs, then close so workers' range loops end
    go func() {
        for j := 1; j <= numJobs; j++ {
            jobs <- j
        }
        close(jobs)
    }()

    // close results once every worker is done
    go func() {
        wg.Wait()
        close(results)
    }()

    for r := range results {
        fmt.Println(r)
    }
}

Study the shutdown sequence, because it’s the part people get wrong: the producer closes jobs when it’s done; workers exit when jobs is drained; a separate goroutine waits for all workers and then closes results; and main’s range ends. Every goroutine has a clear exit.

Pattern 2: pipeline

A pipeline is a chain of stages connected by channels. Each stage receives values, does something, and sends results on. It’s like a Unix pipe: cat | grep | sort.

package main

import "fmt"

func numbers(n int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for i := 1; i <= n; i++ {
            out <- i
        }
    }()
    return out
}

func square(in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for v := range in {
            out <- v * v
        }
    }()
    return out
}

func onlyEven(in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for v := range in {
            if v%2 == 0 {
                out <- v
            }
        }
    }()
    return out
}

func main() {
    for v := range onlyEven(square(numbers(10))) {
        fmt.Print(v, " ")
    }
    fmt.Println() // 4 16 36 64 100
}

Every stage has the same shape: take a receive-only channel, return a receive-only channel, close what you create. That consistency makes pipelines easy to extend.

Pattern 3: fan-out, fan-in

Fan-out means starting several goroutines that read from the same channel to share the work. Fan-in means merging several channels into one. Here’s a reusable merge:

package main

import (
    "fmt"
    "sync"
)

func merge(channels ...<-chan string) <-chan string {
    out := make(chan string)
    var wg sync.WaitGroup
    for _, ch := range channels {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for v := range ch {
                out <- v
            }
        }()
    }
    go func() {
        wg.Wait()
        close(out)
    }()
    return out
}

func source(name string, n int) <-chan string {
    out := make(chan string)
    go func() {
        defer close(out)
        for i := range n {
            out <- fmt.Sprintf("%s-%d", name, i)
        }
    }()
    return out
}

func main() {
    count := 0
    for v := range merge(source("a", 3), source("b", 3), source("c", 3)) {
        _ = v
        count++
    }
    fmt.Println("received", count, "values") // received 9 values
}

Pattern 4: errgroup for “do these in parallel, stop on first error”

This is the pattern I use most in real services: fetch several things at once, and if any fails, cancel the rest and return the error. errgroup does exactly that:

// go get golang.org/x/sync/errgroup
func loadDashboard(ctx context.Context, userID string) (*Dashboard, error) {
    g, ctx := errgroup.WithContext(ctx)
    var profile *Profile
    var orders []Order

    g.Go(func() error {
        var err error
        profile, err = fetchProfile(ctx, userID)
        return err
    })
    g.Go(func() error {
        var err error
        orders, err = fetchOrders(ctx, userID)
        return err
    })

    if err := g.Wait(); err != nil { // first error; ctx is cancelled for the rest
        return nil, fmt.Errorf("loading dashboard: %w", err)
    }
    return &Dashboard{Profile: profile, Orders: orders}, nil
}

Each goroutine writes to its own variable, so there’s no race. g.Wait() returns the first error, and the shared ctx tells the other goroutines to give up early. You can also call g.SetLimit(n) to cap concurrency, which turns it into a simple worker pool.

Best practices from this part

• Guard shared state with a mutex stored next to the data, and use pointer receivers.
• Pass context.Context as the first argument to anything that does I/O or might be slow, and respect ctx.Done().
• Every goroutine needs an exit plan: a closed channel, a cancelled context, or a finished job.
• Close channels from the sending side, and use a separate goroutine with wg.Wait() to close a shared results channel.
• Bound your concurrency. Unlimited goroutines hitting a database is a self-inflicted outage. Use worker pools or errgroup.SetLimit.
• Reach for errgroup when running several fallible tasks in parallel.
• Keep running with -race in tests and CI.

Recap

• sync.Mutex protects shared data. Lock, defer Unlock.
• Channels move data; mutexes guard state. Both are idiomatic.
• context carries cancellation and deadlines through your call chain.
• Worker pools limit concurrency; pipelines chain stages; fan-in merges results.
• errgroup runs tasks in parallel and cancels on the first error.

Try this: combine the worker pool with context.WithTimeout. Give the whole batch 100 ms and make workers stop picking up new jobs once the context is done. Print how many jobs completed.

So far we’ve written lots of functions for specific types. In Part 9, we’ll learn generics, which let you write a function once and use it with many types, safely.

Where to go from here

⬅️ Previous: Part 7: Goroutines, Channels, and select

➡️ Next: Part 9: Generics Without the Headache

The full Go From Zero series

1. Part 1: Why Go, Setting It Up, and Your First Program
2. Part 2: Variables, Types, Control Flow, and Functions
3. Part 3: Arrays, Slices, Maps, and Structs
4. Part 4: Methods, Interfaces, and Composition
5. Part 5: Error Handling the Go Way
6. Part 6: Packages, Modules, and Project Layout
7. Part 7: Goroutines, Channels, and select
8. Part 8: sync, context, and Concurrency Patterns (you are here)
9. Part 9: Generics Without the Headache
10. Part 10: Testing, Benchmarks, and Fuzzing
11. Part 11: Best Practices for Production-Ready Go
12. Part 12: Building a REST API with the Standard Library

If this helped you, share it with a friend who is learning Go, or subscribe to get the next posts by email. Questions are welcome in the comments.