Category: Big Data and Hadoop

In this blog we will be discussing about how to install oozie in hadoop 2.x cluster. First we need to download the oozie-4.1.0 tar file from the below link: Oozie-4.1.0 tar file By default it will be downloaded in the Downloads folder. We need to move into the Downloads...

In this blog we will be discussing about the types of tables in Hive and the difference between them and how to create those tables and when to use those tables for a particular dataset. Hive has two types of tables: Managed table External table Let us see about...

In this blog we will be discussing the loading of data into HBase using Pig scripts. Before going further into our explanation we will be recalling our Pig and HBase basic concepts with the given blog for beginners on Pig and HBase. Link for HBase and Pig blogs: Beginners-Guide-for-HBase...

In this blog we will be discussing about how to convert CSV data to JSON. Let us have the CSV data which available by converting the XML data in our first case. [crayon-57bfa225972ce923274610/] The above command will load the converted XML to CSV file into pig using PigStorage which...

In this blog we will see how to convert JSON format of data into CSV format. We have created our own JSON format data from a CSV file using AVRO file format and we will be using the same JSON data in this blog. You can refer the below...

This blog focuses on converting the XML format of data into CSV format using pig commands. Now we will take a sample XML data. After installing hadoop we get many configuration files in xml format and in this case we are taking hdfs-site.xml as our input data. Our hdfs-site.xml...

In our previous blog Streaming Twitter Data Using Flume we knew about the basics for flume and how to use it for fetching data from twitter. Let’s look another way to use this flume for fetching data from local file system to HDFS. There are three sources for the above...

In this blog post, we have explained about the row-level transactions available in Hive. This post will provide you a good idea of how to implement the row-level transactions on the Hive table. Before beginning with the transactions in Hive, let’s look at the ACID properties, which are vital...

Apache Pig – Introduction Apache Pig is a high-level platform for creating MapReduce programs that will be used with Hadoop.It is a platform for analyzing large data sets, which consist of a high-level language for expressing data analysis programs.The beauty of Pig program’s structure is that they are open...