UNIT-I: Data structures in Java: Linked List, Stacks, Queues, Sets, Maps; Generics: Generic classes and Type parameters, Implementing Generic Types, Generic Methods, Wrapper Classes, Concept of Serialization 

UNIT-II: Working with Big Data: Google File System, Hadoop Distributed File System (HDFS) Building blocks of Hadoop (Name node, Data node, Secondary Name node, Job Tracker, Task Tracker), Introducing and Configuring Hadoop cluster (Local, Pseudo-distributed mode, Fully Distributed mode), Configuring XML files. 

UNIT-III: Writing Map Reduce Programs: A Weather Dataset, Understanding Hadoop API for Map Reduce Framework (Old and New), Basic programs of Hadoop Map Reduce: Driver code, Mapper code, Reducer code, Record Reader, Combiner, Practitioner 

UNIT-IV: Stream Memory and Spark: Introduction to Streams Concepts– Stream Data Model and Architecture, Stream computing, Sampling Data in a Stream, Filtering Streams, Counting Distinct Elements in a Stream, Introduction to Spark Concept, Spark Architecture and components, Spark installation, Spark RDD (Resilient Distributed Dataset) – Spark RDD operations. 

UNIT-V: Pig: Hadoop Programming Made Easier Admiring the Pig Architecture, going with the Pig Latin Application Flow, working through the ABCs of Pig Latin, Evaluating Local and Distributed Modes of Running Pig Scripts, Checking out the Pig Script Interfaces, Scripting with Pig Latin. 

Applying Structure to Hadoop Data with Hive: Saying Hello to Hive, Seeing How the Hive is Put Together, Getting Started with Apache Hive, Examining the Hive Clients, Working with Hive Data Types, Creating and Managing Databases and Tables, Seeing How the Hive Data Manipulation Language Works, Querying and Analyzing data 

Textbooks: -

1. Wiley & Big Java 4th Edition, Cay Horstmann, Wiley John Sons, INC 

2. Hadoop: The Definitive Guide by Tom White, 3 rd. Edition, O’reilly 

Reference Books: -

1. Hadoop in Action by Chuck Lam, MANNING Publ. 

2. Hadoop for Dummies by Dirk deRoos, Paul C.Zikopoulos, Roman B.Melnyk,Bruce Brown, Rafael Coss 

3. Hadoop in Practice by Alex Holmes, MANNING Publ. 

4. Big Data Analytics by Dr. A.Krishna Mohan and Dr.E.Laxmi Lydia 

5. Hadoop Map Reduce Cookbook, SrinathPerera, ThilinaGunarathne 

Software Links: -

1. Hadoop:http://hadoop.apache.org/ 

2. Hive: https://cwiki.apache.org/confluence/display/Hive/Home 

3. Piglatin: http://pig.apache.org/docs/r0.7.0/tutorial.html 

Software Requirements: -

• Hadoop: https://hadoop.apache.org/release/2.7.6.html 

• Java :https://www.oracle.com/java/technologies/javase/javase8u211-later-archive- downloads.html 

• Eclipse : https://www.eclipse.org/downloads/ 

Experiment 1: Week 1, 2: 

1. Implement the following Data structures in Java a) Linked Lists b) Stacks c) Queues d) Set e) Map 

Experiment 2: Week 3: 

2. (i)Perform setting up and Installing Hadoop in its three operating modes: Standalone, Pseudo distributed, Fully distributed (ii)Use web-based tools to monitor your Hadoop setup. 

Experiment 3: Week 4: 

3. Implement the following file management tasks in Hadoop: • Adding files and directories • Retrieving files • Deleting files Hint: A typical Hadoop workflow creates data files (such as log files) elsewhere and copies them into HDFS using one of the above command line utilities. 

Experiment 4: Week 5: 

4. Run a basic Word Count Map Reduce program to understand Map Reduce Paradigm. 

Experiment 5: Week 6: 5. Write a map reduce program that mines weather data. Weather sensors collecting data every hour at many locations across the globe gather a large volume of log data, which is a good candidate for analysis with Map Reduce, since it is semi structured and record oriented. 

Experiment 6: Week 7: 

6. Use Map Reduce to find the shortest path between two people in a social graph. Hint: Use an adjacency list to model a graph, and for each node store the distance from the original node, as well as a back pointer to the original node. Use the mappers to propagate the distance to the original node, and the reducer to restore the state of the graph. Iterate until the target node has been reached. 

Experiment 7: Week 8: 

7. Implement Friends-of-friends algorithm in MapReduce. Hint: Two MapReduce jobs are required to calculate the FoFs for each user in a social network .The first job calculates the common friends for each user, and the second job sorts the common friends by the number of connections to your friends. 

Experiment 8: Week 9: 

8. Implement an iterative PageRank graph algorithm in MapReduce. Hint: PageRank can be implemented by iterating a MapReduce job until the graph has converged. The mappers are responsible for propagating node PageRank values to their adjacent nodes, and the reducers are responsible for calculating new PageRank values for each node, and for re-creating the original graph with the updated PageRank values. 

Experiment 9: Week 10: 

9. Perform an efficient semi-join in MapReduce. Hint: Perform a semi-join by having the mappers load a Bloom filter from the Distributed Cache and then filter results from the actual MapReduce data source by performing membership queries against the Bloom filter to determine which data source records should be emitted to the reducers. 

Experiment 10: Week 11: 

10. Install and Run Pig then write Pig Latin scripts to sort, group, join, project, and filter your data.

 Experiment 12: Week 12: 

11. Install and Run Hive then use Hive to create, alter, and drop databases, tables, views, functions, and indexes