This is the code repository for Apache Spark: Tips, Tricks, & Techniques [Video], published by Packt. It contains all the supporting project files necessary to work through the video course from start to finish.
In this course you'll learn to implement some practical and proven techniques to improve particular aspects of programming and administration in Apache Spark. You will explore 7 sections that will address different aspects of Spark via 5 specific techniques with clear instructions on how to carry out different Apache Spark tasks with hands-on experience. The techniques are demonstrated using practical examples and best practices.
- Compose Spark jobs from actions and transformations
- Create highly concurrent Spark programs by leveraging immutability
- Ways to avoid the most expensive operation in the Spark API—Shuffle
- How to save data for further processing by picking the proper data format saved by Spark
- Parallelize keyed data; learn of how to use Spark's Key/Value API
- Re-design your jobs to use reduceByKey instead of groupBy
- Create robust processing pipelines by testing Apache Spark jobs
- Solve repeated problems by leveraging the GraphX API
To fully benefit from the coverage included in this course, you will need:
To fully benefit from the coverage included in this course, you will need to be
experienced Apache Spark technology.
This course has the following software requirements:
For an optimal experience with hands-on labs and other practical activities, we recommend the following configuration:
OS: Mac Processor: Not Applicable Memory: 4GB or above Storage: 50GB free space
Software Requirements OS: Windows or Mac Browser: Google Chrome Atom IDE, Latest Version Node.js LTS 8.9.1 Installe
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent