Category: Big Data and Hadoop

Hadoop is a tool designed for larger files. But how do you handle small files? This blog gives you a brief overview on solving this problem. There are two primary reasons why small files are problematic in Hadoop: NameNode memory management and MapReduce performance. NameNode Memory Management A common...

In simple words, Kafka is a distributed, partitioned, replicated commit log service. With a unique design It provides the functionality of a messaging system. What does all that mean? First, let’s look at some of the basic messaging terminologies:  Kafka maintains all the feeds of messages in categories called...

In this blog, we will discuss macros in Pig and ways to implement it. It is recommended to go through our previous blog on Beginner’s guide for Pig. Macros were enabled in 0.9 version of Pig. Macros makes the Pig code more modular and makes Pig Latin code shareable....

In this post, we will be looking at how to query files in HDFS using Apache drill. We recommend you to go through our previous post on Installing Apache Drill before going ahead with this post. Note: Drill and Hadoop should be pre-installed in your system. Begin your Drill...

ZooKeeper is a software providing an open source distributed configuration service, synchronization service, and naming registry for large distributed systems.ZooKeeper was a sub-project of Hadoop but is now a top-level project in its own right. ZooKeeper is used by many companies including Rackspace, Yahoo!, Odnoklassniki and eBay as well...