Seems like someone already implemented your idea. https://tinyvg.tech/

Maybe we could use a subset of SVG or PDF?