Description

Hadoop is parallel data processing framework that has traditionally been used to run map/reduce jobs. These are long running jobs that take minutes or hours to complete. Spark has designed to run on top of Hadoop and it is an alternative to the traditional batch map/reduce model that can be used for real-time stream data processing and fast interactive queries that finish within seconds. So, Hadoop supports both traditional map/reduce and Spark.

We should look at Hadoop as a general purpose Framework that supports multiple models and We should look at Spark as an alternative to Hadoop MapReduce rather than a replacement to Hadoop.

Duration

Not Mentioned

Career Option

• SOFTWARE ENGINEERS
• DATA ENGINEERS
• ETL DEVELOPERS
• ASPIRING STUDENTS
• GRADUATES

Opportunies

Not Mentioned

Major Companies Using Spark

• ebay
• Hitachi
• Solutions
• Yahoo!
• Nvidia
• IBM

Course Content

Introduction to Spark

Getting Started

Programming with RDDs

Working with Key/Value Pairs

Loading and Saving Data

Advanced Spark Programming Concepts

Running Spark on a Cluster

Tuning and Debugging Spark

Linking with Spark SQL

Using Spark SQL in Applications

Spark SQL Performance

Spark Streaming

Multiple Sources and Cluster Sizing

24/7 Operation

Streaming UI

Performance Considerations

Garbage Collection and Memory Usage