First off, congrats, this is no small feat, well done.
A question: in my (limited) experience, ERPs are made on the basis of integrations. I'd have thought the best priority order would be data-model first, integration second, everything else third. How do you think about this? What's the goal here?
And secondly, some feedback: It looks like Carbon falls into the same trap as many self-hostable SaaS-like products (including my own!), and that is that software designed for one single hoster is often more complex to deploy and built in a different way, whereas software designed primarily to self-host looks much simpler. As an example, installing Wordpress or Odoo is relatively simple, with basic frontend webserver config and easy to run open source databases. Carbon on the other hand appears to be quite a few different components, with many dependencies, some of which are SaaS products, and uses a database (Supabase) which is itself a whole microservice ecosystem that is a considerable effort to deploy. What's the strategy here? Despite having the skills for it, I'm not sure I'd ever consider self-hosting Carbon, and maybe that's good for Carbon as a business, but it's also less good for the ecosystem.
I was involved in all things ERP years ago. One thing I noticed was that the National Retail Association of the USA (the other NRA) have specification documents which would be perfect for basing both data models and transforms off. It seems all the big players tend to have almost, but not quite compatible models. It's very frustrating. We're talking SAP, Infor, MS, everyone. The amount of glue code I had to write to get various point of sale, accounting, BI, label printing etc software to work is astounding.
I guess that's how they make the money. Sell the platform and profit off the custom code.
This resonates with me a lot. Can I contact you? My email is in my profile.
Feel free, my email is my HN username at gmail. As I mentioned in a sibling comment, I'm not in tech anymore (except for fun) but I'm happy to provide any insights or advice to any questions you may have. I last worked with ERP systems about 10 years ago, but the thing about these systems is that they don't really change, so hopefully my knowledge is still helpful!
As a quick summary, this is mostly all ETL (Extract, Transform, Load). Learn the lowest practical level language you can for each system so you can extract / query exactly the data you need (in SAP it's mostly SQL but there are a bunch of proprietary "languages" and extensions that can help. In MS it's mostly SQL, in Infor it's worth learning some RPG /CL, but only enough to get the data you need). Learn the notification paradigms each system offers (this can be as simple as scheduling jobs, it can be file-system watchers, it can be RPC based, etc. etc. Each system is different, but you need to know when to do stuff with the data. Both on change from external influences, and on completion of whatever job you ran). Thirdly, utilise a modern language for the "Transform" and "Load" phases of the job. I used to use everything from C, C#, Perl, Python, etc. These languages are much easier and much more powerful than something like RPG or pure SQL for a non-maestro. There are people who use complex stored procedures for everything, and that's cool, but it's also a steeper barrier of entry than using a modern, "batteries-included" language.
The reason I initially taught myself Go (13 years ago now) was due to the incredibly powerful and simple data manipulation packages built into it's standard library. It's a damn good language for doing high speed, low latency transformation of lots of data, while being simple and easy to code and deploy. Having said this, the afore-mentioned C, C#, Perl and Python were all working fine before I switched to Go! Use what you know, and what's available in the sytem you're working on.
If you have any questions feel free to send me an email.
That’s exactly how they make the money
True, but I'm not actually in tech anymore. I still have a consultancy business, but I make money doing geo-technical rope-access work now. A bit of a sea-change.
I'm happy to impart my thoughts on this topic for free, but I'm not sure how useful they would be after almost 10 years!
> Carbon on the other hand appears to be quite a few different components, with many dependencies, some of which are SaaS products, and uses a database (Supabase) which is itself a whole microservice ecosystem that is a considerable effort to deploy
Perhaps this could be addressed by providing a Pulumi or Terraform program?
It's not just about the initial deployment, although these may help. If you're running Wordpress, the question "why is it slow" is pretty limited in scope – you need a faster webserver, database, or maybe need those two to be closer to each other. A simplification but not much of one. For this, is it the app server, the task queue, or one of several other components, or is it in the Supabase layer, or is it a dependency that someone else runs... etc.
Figuring out issues get more complex, scaling gets more complex, backup and recovery get more complex, now you need monitoring for it all too, and with many services the complexity multiplies.
All of that complexity is somewhat justified for a team that exists to build and operate the system (i.e. SaaS), but doesn't really work for a team who exist to build something else and have just been tasked with operating this.
There's also the issue of having a dozen moving parts form the system, each developed for their own reasons. So you could find that upgrading one product changes something, say collation for utf-16, which might be entirely reasonable taken in isolation but that the rest of the system is unprepared for.
It is up to you to fend off dozens of small things. Over time that accumulates. It's like python programmers using two dozen pypi dependencies because they've heard nih is bad. Or think nodejs if that's your thing. What you do in practice is you freeze everything in place and then you postpone it all for an increasingly indefinite future. Organizations then periodically need to freeze work for weeks or even months at a time just to keep up with dependencies. This is clearly a lot of work.
Terraform, or even better Ansible, is a good choice to document make those changes over time in a standardized way. But it doesn't make the work go away. The work is there and still has to be done. Those tools also clearly suffers from the same many-moving-parts problem, as soon as you include external modules with their dependencies too, so you have take care not to magnify the problem by accident instead.
This is what system admins and integrators do all day. They will tell you in unison that the best integration is the one that doesn't exist.
I think we can simplify this over time. Everything in our stack is MIT/Apache. I'm keeping my eye on this fastabase project from Drizzle: https://github.com/drizzle-team/fastabase/tree/main
AGPL so I'd wager the idea is to sell this as a service while still allowing self-hosting, perhaps in the hope to gain bug reports, 'free' testing and some leads.