I was having exact same observation, albeit from a bit diffrent perspective: SaaS. This is where as the code tends to be temporary and very domain specific, the data (files) must strive to be boring standards.
The problem today is that we build specific, short-lived apps that lock data into formats only they can read. If you don't use universal formats, your system is fragile. We can still open JPEGs from 1995 because the files don't depend on the software used to make them. Using obscure or proprietary formats is just technical debt that will eventually kill your project. File or forget.
My 10+ year old photo management system [1] relies on the file system and EXIF as the source of truth for my entire photo library.
It’s proven several times over that it’s the correct approach. Abstractions (formerly Google photos, currently Immich) should just be built on top - but these proprietary databases are only for convenience.
For work, I’m having the same experience as the author and everything is just markdown and csv files for Claude Code (for research and document writing).
[1] https://github.com/jmathai/elodie
I know some systems leverage the modern file meta data (extended attributes), but it's clearly not successful enough that folks can use them for an application like this.
Ostensibly, things like MacOS Spotlight can bring real utility and value to the file system, and extended attributes through the sidecar indexing, etc. But Spotlight is infamous for its unreliability.
The other issue with file systems is simply that the user (potentially) has "direct access" to them, in that they can readily move files in and up and around whimsically. The "structure" is laid bare for them to potentially interfere with, or, such as the case with the extended attributes, drag a file to a USB fob, and then copy it back -- inadvertently removing those attributes.
And thats how we end up with everything being stuffed into a SQLite DB.
Yeah, IMO extended metadata attributes are fine for caching data that can be recovered via other means but generally violate the principal of least surprise. For them to be successful a standardized transparent container format or something would be necessary, but at that point the FS abstraction is leaking.
I have your repo starred from a post/comment you made a few weeks ago but haven't had time to actually use/integrate it with my own stuff.
What are your thoughts on XMP sidecar files? I'm torn right now between digital negative + external metadata versus all-in-one image with mutable properties. Portability vs. Durability etc.
I've avoided using XMP sidecars. Mostly because I don't want to have to worry about two files for every photo. And I don't think they're ubiquitously supported like EXIF.
Thanks for starring the repo and let me know if you need any help.
Thanks for sharing, I might have too much NIH syndrome to use it but I'd love to check it out.
Ha! I totally get it. Use it for inspiration though!
The frustrating thing about photo management these days is how every major photo library app/cloud service these days stores every edit / tag / album externally. If you crop a photo, change the taken at date, etc, the original file never gets touched but an external bit of metadata is created. So any time you move platform, all of these edits and your albums are erased.
It is convenient to be able to undo crops or filters, but I wish the industry would standardize so these changes are portable.