I was recently looking at some of the top selected answers for this problem on code wars and noticed a lot of string concatenation happening. While this may be pragmatically ok for small toy examples, if you are going to use string concatenation in a for loop iterating over anything but a small set, you’re probably going to have a bad time.

To highlight this problem, imagine this contrived program that:

  1. Reads in the complete works of Shakespeare (~5.19 MB)
  2. Splits the text into words
  3. Concatenates the words into a single string
  4. Prints the memory usage
func concatString() {
	file, err := os.ReadFile("./shakespeare.txt")
	if err != nil {
		panic(err)
	}
	words := strings.Split(string(file), " ")
	str := ""
	for _, word := range words {
		str += word
	}
	printMemUsage()
}

When I run this via time go run builder.go I get the following output:

Alloc = 60 MiB  TotalAlloc = 1783473 MiB        Sys = 760 MiB   NumGC = 34615
go run builder.go  387.56s user 27.68s system 237% cpu 2:54.60 total`

This took over 6 minutes on a 2021 M1 Max Macbook Pro. You can also see the insane amount of memory 1783473 MiB, that was allocated.

Let’s contrast this approach with strings.Builder:

func buildString() {
	file, err := os.ReadFile("./shakespeare.txt")
	if err != nil {
		panic(err)
	}
	words := strings.Split(string(file), " ")
	var sb strings.Builder
	for _, word := range words {
		sb.WriteString(word)
	}
	printMemUsage()
}

When I run this via time go run builder.go I get the following output:

Alloc = 45 MiB  TotalAlloc = 50 MiB     Sys = 54 MiB    NumGC = 2
go run builder.go  0.17s user 0.13s system 89% cpu 0.332 total

So we’ve gone from 1.7 TiB to 50 MiB - Note that this was over the life of the program and not allocated all at once - and from 6 minutes to 0.17 seconds!

The natural question is, what the heck is going on here?

  1. Strings are immutable in Go, so every time you concatenate a string, you are creating a new string. The old string will need to be garbage collected.
  2. Every new concatenation involves copying the entire content of the existing string into a new string along with the new word. So the longer the string becomes, the more data needs to be copied every time.

When we use a string builder, we are using a buffer without the need to copy the string over and over.

As you can see from the results, the difference can be huge when you start to deal with data sets that are larger than simple toy examples! Oh, and here’s a fun meme related to this topic :-)