Durability just guarantees that you don't return that a write transaction has completed successfully until after all the layers are done writing to disk. fsync is the high level abstraction that file systems implement to mean "this data has actually gone to disk" (although handling errors is a rabbit hole worth reading about). It absolutely has a performance cost which is why applications that can live without durability sometimes get away with it.
If your application can tolerate writes silently failing then you can live without it. But a lot of applications can't, so it does matter.
It depends on if there will be holes and whether you communicate "externally". If none of these are concerns (for WAL and using SQLite locally, none of these are), it is OK.
To elaborate, for a local app that using WAL, if a transaction committed locally, then reverted *along* with everything afterwards, the app restarts, and it will continue to function as expected, no ill-defined states.
If you use WAL within a quorum, sure, durability is a concern and I think you would be better off to have ways above SQLite to maintain that durability rather than relying on fsync solely (your SSD can break).
Also, to add, WAL mode uses checksum to make sure there is no holes, so even your SSD re-orders writes, I think no hole for your writes is a pretty safe assumption.