I've met this unwarranted love for mmap() many times in the developers who never professionally worked on storage projects. Especially common with C++ programmers for some reason. There are people who think they found a "trick" to make I/O go faster and never consider why filesystems or databases don't use it... Like, obviously, those losers who wrote eg. Ext4 never bothered to look at the system interface, right?

On the other hand, if I was ever to advise anyone on how to do I/O when they are working with an (unknown) filesystem... It's really hard. And I'd probably default to saying "do as few tricks as possible" because filesystems today are very elaborate, with a lot of optimizations that are very difficult to predict from user-space. It's quite possible that someone trying to outsmart a filesystem will end up harming themselves in the process.

Doing as few tricks as possible would allow the administrator to configure the filesystem independently of the program writing to it to match the nature of the workload instead of locking the program into a specific pattern of operation that might be impossible to rectify with administrative tools. Not an ideal situation by any means: storage-heavy user-space applications s.a. databases usually do the opposite: they try to optimize for the specific filesystem, its version and quirks... but it takes a lot of effort, obviously.

Libtorrent 2.0 switched exclusively to mmaped read/writes for torrent downloads, which resulted in various performance and especially memory consumption issues on ALL platforms.

For some reason Windows handled increased memory consumption the least gracefully.

Many people continued to use v1.2 which use regular files.

V2.1 ended up using pread/pwrite nowz it's fine now.

The issue continued for 3 years more or less.

https://github.com/arvidn/libtorrent/issues/6667

The principled excuse for mmap is when you're reading all over a file at high performance and you want to avoid either excessive syscalls or double caching. Which sounds like what a torrent program does but evidently it doesn't even work well for them.

[dead]