cargo afl tmin does not work with afl::fuzz for valid inputs

[sequoiaKozak]

My initial approach tried fuzz!(|data: String| { // test code } but running it on sample inputs with cargo afl run < input.txt showed that it was cutting off the input. I switched to using

fuzz!(|bytes: &[u8]| {
      if let Ok(data) = str::from_utf8(bytes) {
          // test code
      }
  }); 

which correctly handles the input, but running cargo afl tmin with non-crashing inputs fails to minimize them at all. My guess is that by instrumenting the string conversion, removing any characters from the input changes the instrumentation output. I can still run cargo afl fuzz, and it also seems to be able to reduce crashing inputs, since I guess it isn't as strict about preserving the full trace.

Is there a way to selectively disable instrumentation for specific parts of source code, as you would with __AFL_COVERAGE(), __AFL_COVERAGE_ON(), __AFL_COVERAGE_OFF()? Or in my particular case, is there a simpler solution of correctly accepting fuzz input as a string which I'm missing?

[sequoiaKozak]

After a bit more poking around, even with a completely empty body, cargo afl tmin does not seem to work with afl::fuzz. While a completely empty fn main() {} results in the input being reduced to zero bytes, which is expected, no minimization is performed at all when afl::fuzz is used, like

fn main() {
    fuzz(true, |_| {
    });
}

The best workaround I have so far is to create a separate version which reads from stdin for minimization, like

    let mut buffer = String::new();
    if let Ok(_) = io::stdin().read_to_string(&mut buffer) {
        // code under test
    }
}

This is capable of replacing some characters, and in some cases removing some blocks of characters. I'm not sure whether instrumentation of read_to_string may be limiting it, but I do get some meaningfully reduced results. My actual fuzzing campaign uses the original version with the fuzz! handle to leverage persistent mode.

Edit: wrong button, didn't mean to close this

[smoelius]

Hi, @sequoiaKozak. From what I can tell, cargo afl tmin seems to be working correctly.

I tested with this program:

fn foo(data: &[u8]) {
    assert!(str::from_utf8(data).is_ok())
}

fn main() {
    afl::fuzz!(|data| foo(data));
}

cargo afl fuzz generated this crashing input:

$ xxd output/default/crashes/id\:000001\,sig\:06\,src\:000000\,time\:0\,execs\:23\,op\:havoc\,rep\:1
00000000: bdbd bdbd bdbd bdbd bdbd bdbd bdbd bdbd  ................
00000010: bdbd bdbd bdbd bdbd bdbd bdbd bd30 0a    .............0.

When I run the following command, the file min is produced:

$ cargo afl tmin -i output/default/crashes/id\:000001\,sig\:06\,src\:000000\,time\:0\,execs\:23\,o
p\:havoc\,rep\:1 -o min -- target/debug/b
...
$ xxd min
00000000: bd                                       .

Does my experiment not capture what you meant to capture?

Also, as you've probably noticed, afl.rs uses Arbitrary for certain types, and I think String is one of them. That could be affecting the analysis.

[sequoiaKozak]

Yes, tmin works for crashing and hanging inputs without issue. The problem I'm experiencing is only for non-crashing inputs specifically, which tmin should also be capable of minimizing, as it does for functions which read from stdin rather than using fuzz.

[smoelius]

OK. I can confirm what you are saying. I created a small C program that operates on strings and afl-tmin did minimize a non-crashing input. Specifically, it changed this:

$ xxd output/default/queue/id\:000000\,time\:0\,execs\:0\,orig\:xs 
00000000: 7878 7878 7878 7878 7878 7878 7878 7878  xxxxxxxxxxxxxxxx
00000010: 0a                                       .

to this:

$ xxd min 
00000000: 3030 3030 3030 3030 3030 3030 3030 3030  0000000000000000
00000010: 30                                       0

However, the cargo afl tmin does not change the input to an analogous Rust program.

Curiously, the following also works (i.e., using the Rust tool to minimize the C program input):

cargo afl tmin -i output/default/queue/id\:000000\,time\:0\,execs\:0\,orig\:xs -o min $PWD/a.out

So the problem must have something to do with how the Rust program is compiled.

I am going to have to look into this more.

[matthiaskrgr]

I saw this too, my none-crashing testcase was reduced to a bunch of zeroes essentially which would totally alter program behavior 😅