It's absolutely doable if you design for it.
The moment you choose to use S3 instead of hosting your own object store, though, you either use AWS because S3 and IAM already have you or spend more time on the care and feeding of your storage system as opposed to actually doing the thing you customers are paying you to do.
It's not impossible, just complicated and difficult for any moderately complex architecture.
There are plenty of compatible S3-like offerings. That's one of the lesser things that tie me to a cloud.
Even on non-AWS projects, I still use S3. I haven't really explored the other options, but if you have opinions or advice I'd love to hear them.
One thing very important, is that I can authorise specific web clients (users) to access specific resources from S3. Such as a document that he can download, but others with the link should not be able to download.
Thank you!
The way I solved auth in my case was just proxying everything through my backend and having that do the auth. I have my own URL scheme and the users never see the URL for the file in S3.
Another way you can do it is generating pre-signed URLs in your backend on each request to download something... but the URL that is generated when you do that is only valid for some small time period, so not a stable URL at all.
In my use case, I needed stable URLs, so I went the proxy route.
Yes, for some use cases proxying is fine. For larger resources or heavily loaded servers (servers I don't want to scale) I prefer to avoid it.
Thank you.