Skip to content Skip to sidebar Skip to footer
Showing posts with the label Hadoop
Pyspark --py-files Doesn't Work

Pyspark --py-files Doesn't Work

Post a Comment
I use this as document suggests http://spark.apache.org/docs/1.1.1/submitting-applications.html sp… Read more Pyspark --py-files Doesn't Work
How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?

How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?

Post a Comment
I'm currently taking a Big Data Class, and one of my projects is to run my Mapper/Reducer on a … Read more How To Run A Mrjob In A Local Hadoop Cluster With Hadoop Streaming?
Exceptions When Reading Tutorial Csv File In The Cloudera Vm

Exceptions When Reading Tutorial Csv File In The Cloudera Vm

Post a Comment
I'm trying to do a Spark tutorial that comes with the Cloudera Virtual Machine. But even though… Read more Exceptions When Reading Tutorial Csv File In The Cloudera Vm
Python Write To Hdfs File

Python Write To Hdfs File

Post a Comment
What is the best way to create/write/update a file in remote HDFS from local python script? I am a… Read more Python Write To Hdfs File
Write Data From Pyspark To Elasticsearch

Write Data From Pyspark To Elasticsearch

Post a Comment
I followed this article to send some data to AWS ES, and I used the jar elasticsearch-hadoop. Here … Read more Write Data From Pyspark To Elasticsearch
Subprocess Popen To Run Commands (hdfs/hadoop)

Subprocess Popen To Run Commands (hdfs/hadoop)

Post a Comment
I am trying to use subprocess.popen to run commands on my machine. This is what I have so far cmdve… Read more Subprocess Popen To Run Commands (hdfs/hadoop)