The point they make is that user data is dynamic, meaning you can't just blindly normalize every json attribute into its own column. The challenge is determining which attributes deserve "shredding" and doing it automatically iiuc
The point they make is that user data is dynamic, meaning you can't just blindly normalize every json attribute into its own column. The challenge is determining which attributes deserve "shredding" and doing it automatically iiuc
Author here. You hit the nail on the head. It's multi-tenant dynamic data. It varies between tenants, even within a single tenant there are large variations across data streams and across time. We decide the layout per physical file we write out using a combination of heuristics and query statistics (we shred attributes that are heavily queried).