fuzz: Extend readme for reproducing and debugging tips

Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Reviewed-by: Andreas Schneider <asn@cryptomilk.org>

tests/fuzz/README.md

@@ -67,3 +67,67 @@ to use none cipher for the key exchange to be plausible.
 
     cp /tmp/ssh_client tests/fuzz/ssh_client_fuzzer_corpus/$(sha1sum /tmp/ssh_client | cut -d ' ' -f 1)
     cp /tmp/ssh_server tests/fuzz/ssh_server_fuzzer_corpus/$(sha1sum /tmp/ssh_server | cut -d ' ' -f 1)
+
+## Debugging issues reported by oss-fuzz
+
+OSS Fuzz provides helper scripts to reproduce issues locally. Even though the
+fuzzing scripts can ran anywhere, the best bet for reproducing is to use
+their container infrastructure. There is a
+[complete documentation](https://google.github.io/oss-fuzz/advanced-topics/reproducing/)
+but I will try to focus here on the workflow I use and libssh specifics.
+
+### Environment
+
+The helper scripts are written in Python and use docker to run containers
+so these needs to be installed. I am using podman instead of docker for
+some time, but it has some quirks that needs to be addressed in advance
+and that I describe in the rejected [PR](https://github.com/google/oss-fuzz/pull/4774).
+You can either pick up my branch or workaround them locally:
+
+ * Package `podman-docker` installs symlink from `/bin/docker` to `/bin/podman`
+ * The directories mounted to the containers need to have `container_file_t`
+   SELinux labels -- this is needed for the `build` directory that is created
+   under the oss-fuzz repository, for testcases and for source files
+ * `podman` does not like combination of `--privileged` and
+   `--cap-add SYS_PTRACE` flags. Podman can work with non-privileged containers
+   so you can just remove the `--privileged` from the `infra/helper.py`
+
+### Reproduce locally
+
+Clone the above repository from https://github.com/google/oss-fuzz/, apply
+changes from previous secion if needed, setup local clone of libssh repository
+and build the fuzzers locally (where `~/devel/libssh` is path to local libssh
+checkout):
+
+    python infra/helper.py build_fuzzers libssh ~/devel/libssh/
+
+Now, download the testcase from oss-fuzz.com (the file under `~/Downloads`)
+and we are ready to reproduce the issue locally (replace the `ssh_client_fuzzer`
+with the fuzzer name if the issue happens in other fuzzer):
+
+    python infra/helper.py reproduce libssh ssh_client_fuzzer ~/Downloads/clusterfuzz-testcase-ssh_client_fuzzer-4637376441483264
+
+This should give you the same error/leak/crash as you see on the testcase
+detail in oss-fuzz.com.
+
+I find it very useful to run libssh in debug mode, to see what happened and
+what exit path was taken to get to the error. Fortunatelly, we can simply
+pass environment variables to the container:
+
+    python infra/helper.py reproduce -eLIBSSH_VERBOSITY=9 libssh ssh_client_fuzzer ~/Downloads/clusterfuzz-testcase-ssh_client_fuzzer-4637376441483264
+
+### Fix the issue and verify the fix
+
+Now, we can properly investigate the issue and once we have a fix, we can
+make changes in our local checkout and repeat the steps above (from building
+fuzzers) to verify the issue is no longer present.
+
+### Fuzzing locally
+
+We can use the oss-fuzz tools even further and run the fuzzing process
+locally, to verify there are no similar issues happening very close to
+existing code paths and which would cause more reports very soon after
+we would fix the current issue. The following command will run fuzzer
+until it finds an issue or until killed:
+
+    python infra/helper.py run_fuzzer libssh ssh_client_fuzzer