Comments on: MapReduce Use Case-Youtube Data Analysis

By: jestin

jestin — Sat, 06 Aug 2016 09:30:02 +0000

Worked like a charm !!!!!

By: Comprehensive list of data science resources [updated May 6, 2016] – WebProfIT Consulting

Wed, 03 Aug 2016 22:49:36 +0000

[…] MapReduce Use Case […]

By: Satyam

Satyam — Mon, 04 Jul 2016 12:31:35 +0000

Hi Karthik,

Thanks for the update.

By default the output of a map reduce program will get sorted in ascending order but according to the problem statement we need to pick out the top 10 rated videos. So to sort it in descending order we have done it using the command.

By: Karthik

Karthik — Fri, 01 Jul 2016 00:22:33 +0000

Nice post.

But I spotted a few errors,
I think in problem 2 explanation line 9 and line 10 are not coded as per explanation. They are not in sync.

I was just wondering if you did send out Rating as key and movie id as value wouldn’t they be sorted by shuffle/sort before they are sent to reducer and you interchange key-value pair in reducer so that you don’t have to get sorted results through a command.

By: Amol

Amol — Wed, 29 Jun 2016 06:25:37 +0000

To sort the Reducer output, the command will be hadoop dfs -cat /user/output/YoutubeData/part-r-00000 | sort -t$’\t’ -k2 -nr | head -5 . This command will take tab as delemeter as Reducer writes its output as tab delemeted by default.

By: FAKHRE ALAM

FAKHRE ALAM — Fri, 29 Apr 2016 17:37:00 +0000

it throws exception when i run the program
Exception in thread “main” java.lang.ArrayIndexOutOfBoundsException: 0
at Top5_categories1.main(Top5_categories1.java:48)

By: FAKHRE ALAM

FAKHRE ALAM — Sun, 24 Apr 2016 11:18:52 +0000

it throws exception when i run the progrmme..

log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Exception in thread “main” java.lang.ArrayIndexOutOfBoundsException: 0
at Video_rating.main(Video_rating.java:59)

By: Satyam

Satyam — Wed, 13 Apr 2016 13:08:53 +0000

This error might be because of the incorrect path of jar file,please check whether you have given correct path for Jar file.

Also ensure that you include main class while exporting the jar file and all your daemons are running in hadoop cluster.

By: Dheeraj

Dheeraj — Thu, 07 Apr 2016 21:21:13 +0000

hi,
i run the command hadoop jar top5.jar /youtubedata.txt /top5_out
but this error is showing-
Exception in thread “main” java.lang.ClassNotFoundException: /youtubedata/txt
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

can you please suggest a solution.

By: 34 External Machine Learning Resources and Related Articles — Dr. Jonathan Jenkins, DBA, CSSBB, MSQA

Thu, 11 Feb 2016 04:50:54 +0000

[…] MapReduce Use Case […]