Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-allocated Jansson json objects between steps in a pipeline. It's extensible in that new middleware can be created. Influenced by bash and OCaml/F#, here is some kind of demo of the syntax:
POST /api/users
|> validate: `
name: string(3..50)
email: email
age?: number(18..120)
team_id?: number
`
|> jq: `{ sqlParams: [.body.name, .body.email, .body.age] }`
|> pg: `INSERT INTO users (name, email, age) VALUES ($1, $2, $3) RETURNING *`
|> result
ok(201):
|> jq: `{ success: true, user: .data.rows[0] }`
validationError(400):
|> jq: `{
error: "Validation failed",
field: .errors[0].field,
rule: .errors[0].rule,
message: .errors[0].message
}`
oh hey Will! long time no see lol, it's been ages. small world.
i think this is on a good track! i like that it's designed to be extensible while still keeping some amount of structure, and that the DSL makes things very compact. how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that?
in general i'd love to see a bunch more DSLs, i think using general-purpose languages for everything is most of the time not actually helpful (cc https://jyn.dev/constrained-languages-are-easier-to-optimize...). i have some vague thoughts about how to make interop between DSLs and the host language easier but i need to think about them some more.
> how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that?
Interpreting in-process. There's a jq C lib, libjq-dev.
I hope you're doing well! I've very much enjoyed your recent blog posts.
I've wanted to do just this. I've used libmicrohttpd, and... it's not my favorite. But if you're going to do it in C then libmicrohttpd is probably the best API to use. You get bonus points for using jq :)
If you're also using jq you might as well ditch Jansson and use jq's `jv` API. I highly recommend it.
One thing that sold me on Jansson is that you can set custom allocators which work very well with the request arena. Once those allocators are set the json objects continue to use the arena even when used in middleware. This makes memory management a cinch!
From what I can tell jq’s C lib doesn’t yet expose a way to set custom allocators.
You can set a custom allocator, but it's global. You could contribute a feature to set thread-local custom allocators -- it'd be quite easy.
Oh right, no, jq has a custom out-of-memory handler. But it should be trivial to extend src/jv_alloc.[ch] to let you set not just a custom nomem handler but also a custom allocator, and then you can use thread-locals in your allocator.
This looks really neat! I typically don't like these DSLs, but this is one I would actually use.