Category: Spark

In this blog, we will discuss a use case involving MovieLens dataset and try to analyze how the movies fare on a rating scale of 1 to 5. We will start our discussion with the data definition by considering a sample of four records. 196 242 3 881250949 186...

In this blog we will see how to access hbase tables using spark. Spark can work on data present in multiple sources like HDFS,Cassandra,Hbase,MongoDB etc. To get the basic understanding of hbase refer our Beginners guide to Hbase According to Spark documentation , “RDDs can be created from Hadoop...

In this post, we will be performing analysis on the Uber dataset  in Apache spark using Scala. The Uber dataset consists of 4 columns. They are dispatching_base_number, date, active_vehicles and trips. You can download the dataset from the below link: https://drive.google.com/open?id=0ByJLBTmJojjzS2c2UktqLW5uRG8 Problem Statement: Find the days on which each basement...

In this blog, we will discuss on the analysis of travel dataset and gain insights from the dataset using Apache Spark. The travel dataset is publically available and the contents are detailed under the heading, ‘Travel Sector Dataset Description’. Based on the data, we will find the top 20...

In this post, we will be analyzing the crimes dataset of New York using SparkSQL. In case you are not familiar with SparkSQL, please refer to our post on Introduction to SparkSQL. Dataset Description: This dataset is available publically, reflects the reported incidents of crime (with the exception of...

In our previous post, we had discussed about the basic RDD operations in Scala. Now, let’s discuss about some of the advanced RDD operations in Scala. Here we have taken two datasets, dept and emp, to work on this operations. The datasets looks like this:       [DeptNo DeptName]                              [Emp_no...

In this blog we will be  taking a famous Tv show dataset i.e., The Daily show and we will be performing analysis on the guests who came to the show. Before going ahead we recommend readers to go through our previous blogs on various publicly available datasets. Youtube Data...