Part of the problem is that the API you're supposed to provide externally depends on git packs which you might have to reconstruct on the fly from such a system.

Exactly. I would love to see that design work, but it doesn't for any realistic workload.

Then the local cache system needs to provide or cache those git packs