Spark – Word count

Word counting using  Spark program from  Windows 7 cmd prompt.  Create a file in the name “SPARK_WORD_COUNT” and save on the C drive . Here we are trying to count the word “HADOOP” from the  saved  file . I added 17 “HADOOP” words in this file  and end of the step , spark program counts 17 “HADOOP” […]

Read More…

Apache Spark – tuning spark jobs-Optimal setting for executor, core and memory

Executor, memory and core setting for optimal performance on Spark Spark is adopted by tech giants to bring intelligence to their applications. Predictive analysis and machine learning along with traditional data warehousing is using spark as the execution engine behind the scenes. I have been exploring spark since incubation and I have used spark core […]

Read More…