Background
This PR implements skew join by deriving hot keys from TopN statistics at planning time. Because hot-key detection depends on TopN, TopN freshness is important: stale or analyze-only TopN can make skew join miss real hot keys or make unreliable decisions.
TODO
- Improve TopN freshness.
Skew join should not only rely on manual ANALYZE. We need to investigate maintaining TopN during append, similar to snapshot statistics freshness, so hot-key statistics are fresh enough for optimizer decisions.
- Support a runtime skew join path.
The current implementation is planner-driven. We should also investigate a runtime version: first distribute rows through normal hash-shuffle channels, detect hot keys while building the hash table, then redistribute hot-key rows again through skew-join channels. This would cover cases where TopN is missing, stale, or not useful for the query.
Background
This PR implements skew join by deriving hot keys from TopN statistics at planning time. Because hot-key detection depends on TopN, TopN freshness is important: stale or analyze-only TopN can make skew join miss real hot keys or make unreliable decisions.
TODO
Skew join should not only rely on manual
ANALYZE. We need to investigate maintaining TopN during append, similar to snapshot statistics freshness, so hot-key statistics are fresh enough for optimizer decisions.The current implementation is planner-driven. We should also investigate a runtime version: first distribute rows through normal hash-shuffle channels, detect hot keys while building the hash table, then redistribute hot-key rows again through skew-join channels. This would cover cases where TopN is missing, stale, or not useful for the query.