My current workflow is to keep a wiki, would you say hydra would replace/complement especially that if you're used to note keeping the wiki way?

Hydra is part of the documentation process imo. Truthfully, the most important stuff that goes in your experiment journal is all those pesky parameters and things that can surprisingly change results.

So I love that hydra uses OmegaConf and I essentially get 3 copies: the experiment config yaml, the wandb log, a dictionary in the checkpoint. Multiple times my dumbass has had to try to match the checkpoint to the wandb log, so the redundancy is incredibly helpful. Sometimes just a library version has unexpected changes on performance and this makes it trivial to trace. The yaml file is more helpful when passing off the code to someone else or releasing to public.

So yeah, I would say that it'll benefit no matter how you document. Use whatever documentation method works for you. Reports can still offer some benefits in just throwing some charts together quickly and organizing but I think you'd still benefit from hydra. It's too easy to lose track of those little things and this helps me automate. But you can also just straight up use OmegaConf or even dictionaries. Whatever works for you.

The real help is logging. So whatever tools help you log, use them. This is just what I benefit from (there's a lot I can talk about too and I'd love to see what others do as well)