When I realized that I was calling openapi-generator to create client side call stubs on non-small service oriented project, I started missing J2EE EJB. And it takes a lot to miss EJB.
I'd like to ask seasoned devs and engineers here. Is it the normal industry-wide blind spot where people still crave for and are happy creating 12 different description of the same things across remote, client, unit tests, e2e tests, orm, api schemas, all the while feeling much more productive than <insert monolith here> ?
I've seen some systems with a lot of pieces where teams have attempted to avoid repetition and arranged to use a single source of schema truth to generate various other parts automatically, and it was generally more brittle and harder to maintain due to different parts of the pipeline owned by different teams, and operated on different schedules. Furthermore it became hard to onboard to these environments and figure out how to make changes and deploy them safely. Sometimes the repetition is really the lesser evil.
> I've seen some systems with a lot of pieces where teams have attempted to avoid repetition and arranged to use a single source of schema truth to generate various other parts automatically, and it was generally more brittle and harder to maintain due to different parts of the pipeline owned by different teams, and operated on different schedules.
I'm not sure what would lead to this setup. For years there are frameworks that support generating their own OpenAPI spec, and even API gateways that not only take that OpenAPI spec as input for their routing configuration but also support exporting it's own.
I see, it's also reminiscent of the saying "microservices" are an organisational solution. It's just that I also see a lot of churn and friction due to incoherent versions and specs not being managed in sync now (some solutions exists are coming though)
> it was generally more brittle and harder to maintain
It depends on the system in question, sometimes it's really worth it. Such setups are brittle by design, otherwise you get teams that ship fast but produce bugs that surface randomly in the runtime.
Absolutely, it can work well when there is a team devoted to the schema registry and helping with adoption. But it needs to be worth it to be able to amortize the resources, so probably best for bigger organizations.
Brb, I'm off to invent another language independent IDL for API definitions that is only implemented by 2 of the 5 languages you need to work with.
I'm joking, but I did actually implement essentially that internally. We start with TypeScript files as its type system is good at describing JSON. We go from there to JSON Schema for validation, and from there to the other languages we need.
> Brb, I'm off to invent another language independent IDL for API definitions that is only implemented by 2 of the 5 languages you need to work with.
Watch out, OpenAPI is now 3 versions deep and supports both JSON and YAML.
If younger me had been told, "one day kid, you will miss working with XML", I'd have laughed.
YAML made me miss JSON. JSON made me miss XML.
anything I could read to imitate that workflow ?
I haven't written anything up - maybe one day - but our stack is `ts-morph` to get some basic metadata out of our "service definition" typescript files, `ts-json-schema-generator` to go from there to JSON Schema, `quicktype-core` to go to other languages.
Schema validation and type generation vary by language. When we need to validate schemas in JS/TS land, we're using `ajv`. Our generation step exports the JSON Schema to a valid JS file, and we load that up with AJV and grab schemas for specific types using `getSchema`.
I evaluated (shallowly) for our use case (TS/JS services, PHP monolith, several deployment platforms):
- typespec.io (didn't like having a new IDL, mixes transport concerns with service definition)
- trpc (focused on TS-only codebases, not multi language)
- OpenAPI (too verbose to write by hand, too focused on HTTP)
- protobuf/thrift/etc (too heavy, we just want JSON)
I feel like I came across some others, but I didn't see anyone just using TypeScript as the IDL. I think it's quite good for that purpose, but of course it is a bit too powerful. I have yet to put in guardrails that will error out when you get a bit too type happy, or use generics, etc.
Can't thank you enough. I'm gonna try these and see.
It's not that we like it, it's just that most other solutions are so complex and difficult to maintain that repetition is really not that bad a thing.
I was however impressed with FastAPI, a python framework which brought together API implementation, data types and generating swagger specs in a very nice package. I still had to take care of integration tests by myself, but with pytest that's easy.
So there are some solutions that help avoid schema duplication.
fastapi + sqlmodel does remove many layers that is true, but you still have other services requiring lots of boilerplate
Having 12 different independent copies means nobody on your 30 people multi-region team is blocked.