About this document
PySpark Installation and Basics Guide by Thái Nguyễn Đức Thông is a document available to read on EtoBox.
This document provides instructions for installing Apache Spark on Windows and introduces PySpark. It discusses installing Java 8, extracting the downloaded Spark zip file, setting environment variables to include Spark, Java and Hadoop bin locations. It also demonstrates running PySpark shell and shows how to create a SparkSession in Jupyter Lab. RDD, DataFrame and DataSet in PySpark are compared. Common SparkSession methods and RDD transformations like flatMap, map and reduceByKey are explained. Finally,
- Author
- Thái Nguyễn Đức Thông
- Language
- EN