Because you want to mock the tools you are going to use in production locally or in CI. Really good reasons to not totally change what you are doing in your app between environments.
The same argument can be made for the other side: the customer can also trivially mount the S3 bucket as a filesystem, if you decide to ship the tool with filesystem support instead.
But is that really something you do in an enterprise application?
What if you write a file with the same name from two servers to the bucket? What about latency? What if the connection / mount fails, how would the application even notice?
Because you want to mock the tools you are going to use in production locally or in CI. Really good reasons to not totally change what you are doing in your app between environments.
> Really good reasons to not totally change what you are doing in your app between environments.
Mocks do that.
If you want to not totally change what you are doing in your app between environments, point it at an actual S3 bucket in all environments.
Because this is about having a local standin for S3, which (in a production deploy) often solves a different problem than a (local) filesystem.
This post does not even mention the S3 API as a replacement for a filesystem.
Because you might be running something that only talks S3 and want to point it at something local.
I like how you got 4 different comments all starting with "Because..." hah!
I can pay Amazon pennies on the dollar to saturate 100Gbps with arbitrarily high concurrency
Sometimes I also want a smaller version for when I don't need that performance but don't want multiple implementations of storage backends
Edit: before someone says NVMe, highly available across 3 fault domains
Because AFAIK the filesystem is at the same time a huge API and designed for a different use case. I can think of the following examples:
- very flat structures: storing hundred of thousands of files in a single directory will fail
- designed for local disks and NFS is a leaky abstraction:
probably many more reasonsMy point is to push the storage abstraction up a level. Then plug S3 or file system into it.
I wouldn’t say this if I hadn’t done this on the PB scale. On a file system! With billions of files!
People are blind to anything other than shitty cloud APIs now. It’s depressing.
Because if you ship a tool with s3 support, the customer can decide if he wants to store in the cloud, or spin up a docker container locally.
The same argument can be made for the other side: the customer can also trivially mount the S3 bucket as a filesystem, if you decide to ship the tool with filesystem support instead.
I did not know you could do that.
But is that really something you do in an enterprise application?
What if you write a file with the same name from two servers to the bucket? What about latency? What if the connection / mount fails, how would the application even notice?
Never tried it, I don't think it fits our needs.