golang: The Sneaky range Pointer

I want to write about an unintuitive phenomenon I encountered today while debugging my code. It has to do with the way memory allocation happens when you range over an array of structs in golang.

Say you have a Deer struct:

type Deer struct {  
    Hooves []Hoof
}

Deer have four hooves, so the above representation seems faithful in terms of memory allocation - hooves belong in the deer, they're a part of it.

Now let's consider the case where we'd like to determine whether the deer is currently hopping. We define a hop as any point in time where the deer has two or more hooves in the air.

For the sake of my example, instead of simply counting how many hooves are mid-air, I will want to do more things with those hooves later, so I want to take that subset into a separate array. Since memory has already been allocated previously for the Hooves (when the Deer was adorably constructed), I would prefer to not allocate it again, meaning I will use an array of Hoof pointers: []*Hoof

Here's what this code would look like:

func (deer *Deer) GetMidAirHooves() []*Hoof {  
    midAirHooves := []*Hoof{}

    for _, hoof := range deer.Hooves {
        if hoof.MidAir() {

            // append the hoof's address
            midAirHooves = append(midAirHooves, &hoof)
        }
    }

    return midAirHooves
}

This is where it may get confusing: when you range over an array of structs, and then reference the current struct's address (&hoof in our example), you're referencing the address of the memory allocated for the iteration itself. This memory has the iterated structs copied into it one after the other, and once the iteration ends, any reference to it will only give you the last struct in the array. The de-facto result is that midAirHooves will contain as many pointers as there are hooves in the air, all of them pointing to the fourth hoof.

The implications is that if you still want to produce a pointer array from a struct array, you'll need to use the index to correctly point to the struct. In our case the loop would look as such:

    for i, hoof := range deer.Hooves {
        if hoof.MidAir() {

            // append the real hoof's address
            midAirHooves = append(midAirHooves, &(deer.Hooves[i]))
        }
    }

Watch out for this tricky behavior when writing your code!