Practical Skew Handling in Parallel Joins

TLDR: This work developed, implemented, and experimented with four new skew-handling parallel join algorithms, one of which, which is called virtual processor range partitioning, was the clear winner in high skew cases, while traditional hybrid hash join was theclear winner in lower skew or no skew cases.

Abstract: We present an approach to dealing with skew in parallel joins in database systems. Our approach is easily implementable within current parallel DBMS, and performs well on skewed data without degrading the performance of the system on non-skewed data. The main idea is to use multiple algorithms, each specialized for a different degree of skew, and to use a small sample of the relations being joined to determine which algorithm is appropriate. We developed, implemented, and experimented with four new skew-handling parallel join algorithms; one, which we call virtual processor range partitioning, was the clear winner in high skew cases, while traditional hybrid hash join was the clear winner in lower skew or no skew cases. We present experimental results from an implementation of all four algorithms on the Gamma parallel database machine. To our knowledge, these are the first reported skew-handling numbers from an actual implementation.