Go 1.18 introduced a number of changes, but the much-anticipated introduction of generics to the language stole the spotlight. Since the release, many people have been talking about generics and when to use them, but 1.18 also introduced native support for fuzzing to the go ecosystem which isn’t getting quite as much attention. Fuzzing is an interesting concept and Go the first language to support it natively, so let’s take a look at what this fuzzing thing is and what it can do for us!
Although it isn’t an entirely new concept, I have no previous experience with fuzzing test techniques. My introduction to fuzzing came with the introduction of it to Go, and the explanation from the Go blog where they explain:
“Fuzzing is a type of automated testing that continuously manipulates inputs to a program to find bugs. Go fuzzing uses coverage guidance to intelligently walk through the code being fuzzed to find and report failures to the user. Since it can reach edge cases which humans often miss, fuzz testing can be particularly valuable for finding security exploits and vulnerabilities.”
Cool! So fuzzing will generate inputs and test cases that we might not think of and execute them extremely quickly. It will write failing cases to a local test data directory so that we can read the input that triggered a failure and turn that input into a unit test.
ExampleThe Go article gives an outline of requirements and suggestions to follow when identifying fuzzing targets. Important to note here is that the function under test should be small, fast, deterministic, and not rely on an external state and that the fuzzing arguments can only be of the following types:
- string, byte
- int, int8, int16, int32/rune, int64
- uint, uint8/byte, uint16, uint32, uint64
- float32, float64
Fuzz tests look and behave similar to normal Go unit tests, but with
f *testing.F replacing
t *testing.T as the test argument, and
t.Run. We also need to seed the fuzz engine with some values to set a baseline. We do this via the
f.Add method. The
f.Add method can take an arbitrary number of arguments, but the number and types must correspond to the arguments in the f.Fuzz method. I worked through the Go blog tutorial which uses the example of a string reversal function and shows how we can incrementally improve it using the bugs found via the fuzz tests. I also tried out writing some simple fuzz tests myself to get a better understanding of the bits and pieces involved
If you execute the test like a normal Go test, it will behave as such. It will use the seed corpus as a test table and tell you if there is a failure.
We can launch our fuzz test with the command:
go test -fuzz=FuzzRepeat
We can optionally set a time limit for our fuzzing by providing the optional
go test -fuzz=FuzzRepeat -fuzztime 30s
This is useful for integrating into CI pipelines, otherwise, it will run indefinitely or until it encounters an error.
After running our fuzz test we see the below output:
There isn’t a lot happening for these tests, as the code we are testing is extremely simple. Let’s coerce our function into a panic and see how the fuzzer handles it. I added the following if statement to the implementation of the
I then ran the fuzz test again but without a time limit
go test -fuzz=FuzzRepeat. This time I don’t use a time limit because I want it to run until it encounters the failing test input.
Nice! In just 1 minute and 6 seconds, the fuzzer ran the function 10729739 times and found that the input “GeekSpace9” caused a panic. It then created a file locally in
testdata/fuzz/FuzzRepeat containing the following:
go test fuzz v1 string("GeekSpace9") int(-32)
Subsequent runs of the FuzzRepeat test use this testdata as baseline coverage to catch any regressions. This basic example is available to run at this repository.
I wanted to see a more complex scenario for fuzz testing, and fortunately, the Go blog links to the Go Fuzzing Trophy Case where some real-world bugs caught with fuzzing can be reviewed. The go-zero framework also uses some fuzz tests which are a bit more complicated and were interesting to read (for example, they fuzz the number of workers in their mapreduce tests).
Now that I have some foundational knowledge on fuzzing, I want to see if I can find an area where it would benefit us in the applications we are developing at Geek Space 9. Adding fuzzing tests to our pipelines to ensure the security and reliability of our microservices may help us identify unforeseen issues. It seems from the examples in the Go Fuzzing Trophy case that fuzzing is a particularly useful tool in identifying improper handling of non-utf8 encoded string values, so any exposed interface that accepts string values and performs manipulation or validation of that string will be a good candidate for fuzzing.