> You never need to run any sort of complex SQL query on an image file format for instance.
Sure you will. Plenty of features that don't exist, or are implemented badly, because you can't easily do it.
Quick mental translation table: if you think "iterate over every ..." or a `for` loop, that's your SELECT query. If you think about `if` conditions, that's the parts that go after FROM clause.
In order to have any advantage from this, you would have the added complexity of splitting your file format into tables that can be queried in a useful manner. However, for an image file format, you most likely need to hold the entire definition in memory at all times anyway. Assuming that’s the case, doesn’t XPath get you there most of the way (assuming XML), with _way_ less complexity?
Image data is just binary blobs. You aren't splitting that into channel columns or anything. But an image file for an editor like Gimp isn't one image blob. It's dozens or hundreds of them - one or more per layer - along with tons of associated metadata at every level.
All that tends to fit sensible schemas and managing it is what SQLite shines at.
I don’t see how this addresses my point that zipped XML gives you the same thing, but simpler. I understand that a GIMP file is many images, so that makes a zip feel like a great fit to me. The only advantage I see for using a full-blown DB is ensuring consistency with references, which admittedly is a plus. But beyond that, what do you gain?
You're not:
- Continuously parsing and writing and reparsing text, 90% of which is useless (that's the JSON/S-expressions vs XML argument)
- Forcing a diverse relational structure to fit a tree hierarchy, hand-writing all the logic that manages representation change - either explicitly, at serialization boundary, or implicitly, in every single access operation you're doing to refer to some data;
- Or worse, using an off-the-shelf, generic object/XML mapper, in which case you just compound the bloat even more.
SQLite is one of the single most battle-tested and ubiquitous piece of software in the history of mankind. Anything "simpler" you're going to pick up is much more likely to be buggy and broken, and will definitely be orders of magnitude slower.
SQLite has it's own issues - for example it doesn't support checksums. Quite bizarre for a database file format where integrity is supposedly highly valued. Saying it's the job of the filesystem doesn't help when all major OSes don't enable checksums in their default filesystem.
With a zip file at least you know your file was corrupted.
https://avi.im/blag/2024/sqlite-bit-flip/
SQLite is fast, but it won't be faster than a hot loop in C. Having the loop construct dictated by the file format seems bad. For images it seems more reasonable to have them in-memory, except for huge image edge cases.
Images are the red herring. Pixel data is best read in hot loops in C, but that would be stored as blobs in SQLite anyway.
It's all the metadata around the image that's interesting. Images have layers, dozens or hundreds of them (this literally scales with how good your software is at handling those - the faster, and more powerful layer UX is, the more they get used). Some are pixel layers, other are effect layers, text layers, vector layers. Layers have metadata - names, sizes, colors, tags, types, special effects, and a bunch of other stuff I don't know because I don't use that 80% of features of GIMP/Photoshop/Affinity.
Then you have document level metadata, UI-specific metadata, etc. Also undo history. A lot of that is relevant to the work on images themselves, and changes in realtime, and can get even more useful if querying it wasn't such a PITA.
That - not the binary pixel blobs - is the selling case of using SQLite as application data format.