Learn. Do. Earn.

Category: Big Data and Hadoop

Streaming Twitter Data using Kafka

Streaming real time twitter data using kafka
In this post, we will be discussing how to stream Twitter data using Kafka. Before going through this post, please ensure that you have installed Kafka and Zookeeper services in your system. You can refer to this post for installing Kafka and this one for installing Zookeeper. Streaming Twitter...
Read More
Leave a Comment

Implementing Custom Output Format in Hadoop

Custom-output-format-in-Hadoop
In this post, we will be discussing how to implement Custom output format in Hadoop. Here we will implement XML output format, which converts all the output keys and values into XML format. Let’s implement a word count program in MapReduce and write a custom output format which stores...
Read More
Leave a Comment

Cassandra Installation on CentOS

Cassandra Installation
If you need scalability and high availability without compromising performance, then the Apache Cassandra Database is the right choice for you. Cassandra is a distributed database; initially developed by Facebook, it later on came under the Apache forum. In this post, we will be cover basic topics as an...
Read More
Leave a Comment

Introduction to Apache Maven

Introduction to Maven
In this blog we will be discussing about maven, how it works and why it is famous for. Let’s come from the History of Maven. History of Maven The word maven means “accumulator of knowledge” in Yiddish. Maven began its life in the Jakarta Alexandria project.Though Maven started in...
Read More
Leave a Comment

Parquet File Format Hadoop

Parquet File format in Hive
In this blog we will discuss about most widely used file format in Hadoop ecosystem Parquet Parquet, an open source file format for Hadoop. Parquet stores nested data structures in a flat columnar format .Compared to a traditional approach where data is stored in row oriented approach, parquet is...
Read More
Leave a Comment

Ambari Installation Guide – (part I)

1- Ambari Installation Guide- part 1
What is Ambari ? The Apache’s Ambari is a Hadoop management tool aimed at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its restful APIs (Definition given by Apache Ambari) How...
Read More
Leave a Comment

Custom Input Format in Hadoop

Custom input format in Hadoop
In this post, we will be looking at ways to implement custom input format in Hadoop. For doing this, we have taken the Titanic data set as an example and have implemented the following problem statement. Problem Statement: Find out the number of people who died and survived, along...
Read More
Leave a Comment

Counters in MapReduce

Counters-in-MapReduce
In this post we will be discussing about how counters work in hadoop and how to use our own custom counters to collect statistics from a job that is being done using MapReduce. Apart from this we will also discuss the points mentioned below: How to display counters to...
Read More
Leave a Comment

SBT in CentOS

SBT-installation-guide
In this post we will be discussing about the set up of SBT(simple build tool) in cent os and will be running a sample program using SBT. Let’s start with the features of SBT. Sbt is an open source build tool for Scala and Java projects, similar to Java’s...
Read More
Leave a Comment
© Copyright 2016.  ACADGILD.