Summary: Spark (and Pyspark) use map, mapValues, reduce, reduceByKey, aggregateByKey, and join to transform, aggregate, and connect datasets. Each function can be stringed together to do more complex tasks.
Summary: Always check your numbers with smaller, simpler queries and figures. Use total sales as a reality check for comparison to sales queries. When creating models, compare performance to a simpler model. Don’t assume complexity equals accuracy. Be prepared to compare against existing “gold standard” models.
Summary: Mixes code and concepts very well. Consistent format between each topic chapter. Great for R newbies and R pros looking for insight into a specific model or package.
Summary: Dean Wampler from Lightbend presented at the Direct Supply MSOE offices on Tuesday, 4/5/2016. Dean covered a high-level overview of Spark and its benefits (business logic is focus of code and it’s faster). Those wanting to learn more should pick up Learning Spark at O’Reilly books.