I certainly do appreciate that the file format internals are so well documented here. It really reveals a lot of information about the inner workings of sqlite itself. I highly recommend reading it; I actually saved a copy for a rainy day sometime and it was very insightful and absolutely influenced some design decisions using sqlite in the future.
The format itself is a U.S. federal standard, and cannot be changed. That has advantages and drawbacks.
https://www.sqlite.org/locrsf.html
I assume the SQLite team could increment the version to 4 if they really needed to, and leave the LOC to update (or not) their recommendation, which specifies version 3.
Very true.
However, a significant fraction of the current installed base would not upgrade, requiring new feature development for both versions.
The test harness would also need implementations for both versions.
Then the DO-178B status would would need maintenance for both.
That introduces significant complexity.
Compared to the amount of SQLite database files in the world only few are shared between different applications. If there is an upgrade path most won't notice. The bigger issue imo is API and SQL dialect compatibility.