> This is particularly frustrating if functions are sometimes async, like lazy loaders or similar cache things.

This is a solved problem in C#. You can use ValueTask<T> instead of Task<T> and no promise will be allocated if it never awaits.