Have you considered something like iceberg tables?
Yes, but parquet hates small files.
You can't compact? i.e. iceberg maintenance
Yes, but parquet hates small files.
You can't compact? i.e. iceberg maintenance