The Spark SQL module was introduced to reduce those limitations, and while the addition of SQL capabilities expanded what Spark can do, the performance still came up short by “an order of magnitude” ...