I've found most of the differences to come not from the socket API, or the logical behavior, or CLI differences. But instead from assumptions Docker makes, that it's running rootful, when Podman will not (by default).
As such, most of the fixes for Podman/Docker incompatibilities is just addressing that assumption with a few extra flags on the Podman commands to change how the user namespace maps between the container and the host, etc etc.
This has been my experience too. And that conclusion, I've found that if I really don't want to deal with such quirks, I can indeed literally just run the same docker compose file in rootful mode, and it works.
Making the changes required to run under rootless is often very simple.
There are also different CLI flags (I recall something related to building and pushing multi-arch containers) and different features supported in compose.yml. Even rootful Podman won't be fully Docker-compatible.