Welcome to Spark Python API Docs!

Contents:

• pyspark package
  • Subpackages
  • Contents
• pyspark.sql module
  • Module contents
• pyspark.streaming module
  • Module contents
• pyspark.mllib package
  • Submodules
  • pyspark.mllib.classification module
  • pyspark.mllib.clustering module
  • pyspark.mllib.feature module
  • pyspark.mllib.linalg module
  • pyspark.mllib.random module
  • pyspark.mllib.recommendation module
  • pyspark.mllib.regression module
  • pyspark.mllib.stat module
  • pyspark.mllib.tree module
  • pyspark.mllib.util module

Core classes:

pyspark.SparkContext

Main entry point for Spark functionality.

pyspark.RDD

A Resilient Distributed Dataset (RDD), the basic abstraction in Spark.

Indices and tables

• Search Page