Predicate and Projection Pushdown

Parquet In Depth covered a detail almost in passing: the footer holds per-column statistics, read before any actual data. This article is about what that detail makes possible for a column-oriented format like Apache Parquet — the single biggest reason a well-organized file is fast to query at all.

Projection pushdown: read only the columns the query needs#

Projection is query-planning terminology for "which columns are actually selected." Pushing that down means the storage layer — not the query engine after the fact — only reads the column chunks a query actually references:

SELECT customer_id, amount FROM orders WHERE amount > 1000;

Against a Parquet file, this reads the customer_id and amount column chunks from every row group and skips order_id, status, and every other column entirely — not filtered out after reading, never read in the first place. This is the direct payoff of the column-oriented layout covered in Row-Oriented vs. Column-Oriented Storage.

Predicate pushdown: skip whole row groups using statistics alone#

Predicate pushdown goes further: using each row group's min/max statistics from the footer, a reader can determine a row group cannot possibly contain a matching row — and skip decompressing it entirely, without reading a single one of its actual values:

Query:  WHERE order_date >= '2026-06-01'
Row group 1 stats: order_date min=2026-05-01, max=2026-05-15                    -> max is before the filter's lower bound: SKIP ENTIRELY
Row group 2 stats: order_date min=2026-05-28, max=2026-06-10                    -> range overlaps the filter: must actually read this one

For a file sorted or naturally clustered by the filtered column, this can eliminate the overwhelming majority of a scan before a single page is decompressed — the query optimization technique responsible for most of the difference between "scan the whole table" and "touch a tenth of it," achieved entirely from metadata, before any real I/O for the skipped groups happens at all.

Where pushdown quietly stops working#

Wrapping the filtered column in a function or expression breaks predicate pushdown, because the stored min/max describes the raw column, not the transformed value:

-- Pushdown-eligible: compares the raw column directlyWHERE order_date >= '2026-06-01'
-- Not pushdown-eligible: the stored statistics describe order_date,-- not UPPER(status) — every row group must be read to evaluate thisWHERE UPPER(status) = 'SHIPPED'

The second query still returns the correct answer — it just cannot use the footer's statistics to skip anything, because those statistics were never computed against UPPER(status). A query that filters on a raw column, unmodified, keeps every optimization this article describes available; one that filters on a computed expression usually gives it up without any error or warning that it happened.

Partition pruning: the same idea one layer up#

Partitioning Files on Disk covers a directory-level version of exactly this mechanism — skipping entire files, based on their path, before even opening them to check a footer. Predicate pushdown works inside one file; partition pruning works across a whole directory tree of them, and the two compose: prune directories first, then push predicates down within whatever files remain.

Common mistakes#

SELECT * when only a few columns are actually needed#

Every unreferenced column chunk still gets read, decompressed, and discarded. Naming the columns a query actually needs is a small change with a direct, often large, effect on how much data the engine has to touch.

Filtering on a function of a column instead of the column itself#

WHERE UPPER(status) = 'SHIPPED' and WHERE status = 'SHIPPED' can return the same rows and cost very different amounts to compute — the second keeps predicate pushdown available, the first silently gives it up.

Assuming pushdown always happens without checking#

Not every query engine, and not every combination of file format and filter shape, supports pushdown the same way. Checking an engine's actual query plan — most show explicitly whether a filter was pushed down — is the only way to confirm it happened rather than assume it did.

Further reading#

• Query optimization — the general field this technique belongs to.
• Apache Parquet documentation — how statistics are computed and stored per column chunk.
• Parquet In Depth — the row group and footer structure this article's skipping mechanism depends on.

See also#