Helps use all of the available bandwidth by not blocking (otherwise idle) worker threads waiting for data to arrive. Requests and processing are managed independently, so you essentially get better utilization & improved performance.
highest impact is when there is network overhead, like accessing them over S3/HTTP/S, and you're accessing many files as part of a query, like using a glob `-*.parquet` pattern. I imagine the same applies when you're doing local I/O, but not as pronounced of an improvement due to lower latency.
Helps use all of the available bandwidth by not blocking (otherwise idle) worker threads waiting for data to arrive. Requests and processing are managed independently, so you essentially get better utilization & improved performance.
highest impact is when there is network overhead, like accessing them over S3/HTTP/S, and you're accessing many files as part of a query, like using a glob `-*.parquet` pattern. I imagine the same applies when you're doing local I/O, but not as pronounced of an improvement due to lower latency.
yes, makes sense because bandwidth is less likely to be the bottleneck