Big Data Analytics Online Training

Big Data Analytics Course Syllabus

Big Data Analytics Course Content

1. Overview of Big Data

This includes topics such as history of big data, its elements, career related knowledge, advantages, disadvantages and similar topics.

2. Using Big Data in Businesses

This module should focus on the application perspective of Big Data covering topics such as using big data in marketing, analytics, retail, hospitality, consumer good, defense etc.

3. Technologies for Handling Big Data

Big Data is primarily characterized by Hadoop. This module cover topics such as Introduction to Hadoop, functioning of Hadoop, Cloud computing (features, advantages, applications) etc

4. Understanding Hadoop Ecosystem

This includes learning about Hadoop and its ecosystem which includes HDFS, MapReduce, YARN, HBase, Hive, Pig, Sqoop, Zookeeper, Flume, Oozie etc.

5. Dig Deep to understand the fundamental of MapReduce and HBase

This module should cover the entire framework of MapReduce and uses of mapreduce.

6. Understanding Big Data Technology Foundations

This module covers the big data stack i.e. data source layer, ingestion layer, source layer, security layer, visualization layer, visualization approaches etc.

7. Databases and Data Warehouses

This module should cover all about databases, polygot persistence and their related introductory knowledge

8. Using Hadoop to store data

This includes an entire module of HDFS, HBase and their respective ways to store and manage data along with their commands.

9. Learn to Process Data using Map Reduce

This emphasizes on developing simple mapreduce framework and the concepts applied to it.

10. Testing and Debugging Map Reduce Applications

After the applications are developed, the next step is to test and debug it. This modules imparts this knowledge.

11. Learn Hadoop YARN Architechture

This module covers the background of YARN, advantages of YARN, working with YARN, backward compatibility with YARN, YARN Commands, log management etc.

12. Exploring Hive

This modules introduces you with all the necessary knowledge of Hive.

13. Exploring Pig

This modules introduces you with all the necessary knowledge of PIG.

14. Exploring Oozie

This modules introduces you with all the necessary knowledge of Oozie.

15. Learn NoSQL Data Management

This modules covers all about NoSQL including document databases, relationships, graph databases, schema less databases, CAP Theorem etc.

16. Integrating R and Hadoop and Understanding Hive in Detail

This module introduces you to RHadoop, ways to do text mining and related knowledge

Module 0. Introduction and Setup:

• How to start Spark and Zeppelin services in Ambari
• How to login to Spark using Python and Scala

Module 1. Spark Architecture:

• What is Apache Spark?
• Spark processing (Jobs, Stages, Tasks)
• Spark components (Driver, Context, Yarn, HDFS, Workers, Executors)

Module 2. Getting Started with RDDs:

• Running queues in Python, Scala and Zeppelin
• Queries using most popular Transformations and Actions
• Creating RDDs

Module 3. Pair RDDs:

• Difference between RDDs and Pair RDD
• 1 Pair Actions, 1 Pair Transformations and 2 Pair Transformations

Module 4. Spark SQL:

• Working with DataFrames and Tables and DataSets
• Catalyst optimizer overview

Module 5. Spark Streaming):

• Working with DStreams
• Stateless and Stateful Streaming labs using HDFS and Sockets

Module 6. Visualizations using Zeppelin:

• Creating various Charts using DataFrames and Tables
• How to create Pivot charts and Dynamic forms

Module 7. Spark UI

• Overview of Job, Stage and Tasks
• Monitoring Spark jobs in Spark UI

Module 8. Performance Tuning:

• Caching, Checkpoint, Accumulators and Broadcast Variables
• Hashed Partitions, Tungsten, Executor memory and Serialization

Module 9. Spark Applications

• Creating an application via spark-submit
• Parameter configurations (number executors, driver memory, executor cores, etc.)

Module 10. Spark 2.0 Machine Learning (ML)

• How ML Pipelines work
• Making Predictions using Decision Tree

Module 1. Datasets and Catalog:

• What is a Dataset?
• When to use which object
• Encoders and semi-structured data
• Common ways to create DS
• Cannot create DS these ways
• Casting DS and convert DS to DF to RDD
• Review questions: Datasets / Catalog
• Dataset versus SQL/DataFrames
• Serialization performance using Encoders
• Dataset caching
• Creating DS from an RDD
• Casting DS and convert DS these ways
• Hive list Catalog
• In Review: Datasets / Catalog

Module 2. Catalyst and Tungsten functionalities:

• Before we begin: Open Zeppelin note
• DataFrames, Datasets and Views use Catalyst / Tungsten
• Catalyst optimizer overview
• Catalyst: Join on 2 Spark views demo
• But RDDs can’t use Catalyst
• Loading data in Spark 2.x and Catalyst
• Loading data in Spark 2.x and Catalyst
• Load data (old way), then join
• Execution Plan from ‘old way’ loading
• DataFrameReader: Load / Execution plan
• Dropping hints to Catalyst
• Catalyst: column pruning demo
• Catalyst: Column (& Partition) pruning
• Catalyst: Predicate pushdown concepts
• Tungsten overview
• Binary processing
• Improved Memory usage
• Improved caching demo
• Whole-stage code gen
• Whole-stage code gen demo
• Whole-stage code gen Vectorization
• Review questions: Catalyst / Tungsten
• In Review: Catalyst / Tungsten

Module 3. Performance Tuning:

• 2 types of Machine Learning
• How Models are Created
• Four Common MLlib functions
• What is Supervised Learning?
• Spark Superbised Learning Workflow
• Unsupervised Learning
• RDD – Machine Learning (MLlib)
• KMeans scenario
• Load data
• Create Model and Predict
• Compare Actual to Predict
• Collaborative Filtering (CF) recommender
• Lab: Will We like Star Wars?
• Classification Functions (Supervised)
• Classification uses LabelPoint
• CASTing X-var and Y-vars for LabelPoint
• Logistic regression, Support Vector Machines, NaiveBayes and Decision Tree (Supervised)
• ML Pipeline terminology
• How ML Pipeline works
• Cleaning the data
• Train ML pipeline – The Big Picture
• Imprving the Model
• Lab: Predict Titanic Survivors (Random Forest)
• Review Questions: Machine Learning
• In Review: Machine Learning
• But wait, there’s more (for MLlib) (Appendix)
• Linear Regression on scenario (Supervised)