Iterators in GoLang
Table of Contents
Iterators in GoLang were introduced in Go 1.23, and it was about time.
Many programming languages like Python, Ruby, and JavaScript have iterators, which are powerful tools for iterating over collections.
This post explores how to use iterators in GoLang, specifically demonstrating their use with the for loop and an example from Advent of Code 2024.
Why to use an iterator
Iterators provide a memory-efficient way to process sequences of data by generating values one at a time, only when needed. This “lazy evaluation” approach offers several benefits over returning complete slices.
Consider a simple example to illustrate the concept:
type User struct {
ID int
Name string
}
// Traditional approach - returns full slice
func GetAllUsers() []User {
var users []User
// Imagine fetching 1M users from database
for i := 0; i < 1_000_000; i++ {
users = append(users, User{ID: i, Name: fmt.Sprintf("User%d", i)})
}
return users
}
Then you would iterate over the users slice like this:
for _, v := range GetAllUsers() {
fmt.Println("👤 user", v)
}
The problem with this approach is that you have to wait for the entire slice to be generated before you can start processing the data.
Besides the waiting, we need to allocate memory for the entire slice, which can be a problem if the slice is large.
Some cases iterators might be useful (not exhaustive list):
- Processing large datasets where you might not need all elements
- Working with resource-intensive operations
- Dealing with memory constraints
- Implementing infinite sequences
Iterators to the rescue
Go 1.23 introduced a new way to iterate over a collection of items using the for loop.
The for loop can now iterate over an iterator, which is a function that returns the next element in the sequence.
The signature of an iterator function is:
func(yield func(T) bool)
Where T is the type of the elements in the sequence, and yield is a function that returns the next element in the sequence.
Notice that yield is conventionally named like this, but you can name it whatever you want.
To re-write our previous example, the GetAllUsers function would become an iterator:
func iterateAllUsers(yield func(User) bool) {
for i := 0; i < 1_000_000; i++ {
if !yield(User{ID: i, Name: fmt.Sprintf("User%d", i)}) {
return
}
}
}
Now every time you call yield, the for loop will return the next element in the sequence.
for user := range iterateAllUsers {
fmt.Println("👤 user", user)
}
The for loop will call the iterateAllUsers function repeatedly until it yield returns false.
You can think of the yield as a function that says “give me more values”, and our iterator can be finalised with a
return or a break when we are done looping (example our for loop having a break statement)
Standard library iterators
The Go standard library provides a few iterator functions and helpers that you can use to iterate over collections of items, as now itearators are a first-class citizen in GoLang.
Single value iterators
If our iterator returns a single value, we can use the iter.Seq type to create an iterator function.
type Seq[V any] func(yield func(V) bool)
This signature says that we should expect only one value yielded at a time.
Here is an example of how to use the iter.Seq type to create an iterator that returns the users one by one:
import "iter"
func iterateAllUsers() iter.Seq[User] {
return func(yield func(User) bool) {
for i := 0; i < 1_000_000; i++ {
if !yield(User{ID: i, Name: fmt.Sprintf("User%d", i)}) {
return
}
}
}
}
And then we would use it as usual with our for loop:
for user := range iterateAllUsers() {
fmt.Println("👤 user", user)
}
Two value iterators
If our iterator returns two values, we can use the iter.Seq2 type to create an iterator function.
type Seq2[K, V any] func(yield func(K, V) bool)
Here is an example of how to use the iter.Seq2 type to create an iterator that returns the users one by one,
alongside the index of the loop:
import "iter"
func iterateAllUsers() iter.Seq2[int, User] {
return func(yield func(int, User) bool) {
for i := 0; i < 1_000_000; i++ {
if !yield(i, User{ID: i, Name: fmt.Sprintf("User%d", i)}) {
return
}
}
}
}
And then we would use it as usual with our for loop:
for idx, user := range iterateAllUsers() {
fmt.Println("👤 user", user, "at index", idx)
}
Usually the first value is the index of the loop, and the second value is the actual element, to follow the convention of looping Maps or Slices.
Slice iterators
Let’s explore some examples of using iterators with slices. I’ll illustrate this with a problem from the Advent of Code 2024.
Here’s a slightly modified challenge:
Find if the word "XMAS" exists in a slice of strings,
either forward or backward.
There are many ways to tackle this, but I’ll show you how to use iterators to solve it.
This is not the most efficient way to solve this problem, but it’s a good example to show you how to use iterators.
package main
import (
"fmt"
"iter"
"slices"
)
// checkMAS receives an iterator and checks
// is elements are the same as the string XMAS in order
func checkMAS(seq iter.Seq2[int, string]) bool {
var XMAS = []string{"X", "M", "A", "S"}
var idx = 0
for _, val := range seq {
if val != XMAS[idx] {
return false
}
idx++
}
return true
}
func main() {
var workingExampleForward = []string{"X", "M", "A", "S"}
var workingExampleBackward = []string{"S", "A", "M", "X"}
var nonWorkingExample = []string{"X", "M", "A", "X"}
// Will print true
fmt.Println(checkMAS(slices.All(workingExampleForward)) || checkMAS(slices.Backward(workingExampleForward)))
// Will print true
fmt.Println(checkMAS(slices.All(workingExampleBackward)) || checkMAS(slices.Backward(workingExampleBackward)))
// Will print false
fmt.Println(checkMAS(slices.All(nonWorkingExample)) || checkMAS(slices.Backward(nonWorkingExample)))
}
In this example, we use the slices.All and slices.Backward functions to create iterators that return the elements of the slice in order and in reverse order, respectively.
This is convenient because we can use the same checkMAS function to check if the word “XMAS” exists in the slice, either forward or backward without modifying the function.
Conclusion
Iterators in GoLang are a powerful tool that allows you to iterate over a collection of items one at a time, only when needed.
In this post, we explored how to use iterators in GoLang and how to create your own iterators using the for loop.
We also saw how to use the iter.Seq and iter.Seq2 types to create iterators that return one or two values, respectively.
If you have other examples of how to use iterators in GoLang, feel free to share them in the comments below.
comments powered by Disqus