MIT-Stanford Project Uses LLVM to Break Big Data Bottlenecks

By InfoWorld

Written in Rust, Weld can provide orders-of-magnitude speedups to Spark and TensorFlow.

The more cores you can use, the better — especially with big data. But the easier a big data framework is to work with, the harder it is for the resulting pipelines, such as TensorFlow plus Apache Spark, to run in parallel as a single unit.