option (DataFrameReader)

Adds an input option for the underlying data source.

Syntax

option(key, value)

Parameters

Parameter Type Description
key str The option key.
value str, int, float, or bool The option value.

Returns

DataFrameReader

Examples

Read a CSV file with the nullValue option set.

import tempfile
with tempfile.TemporaryDirectory(prefix="option") as d:
    df = spark.createDataFrame([{"age": 100, "name": "Alice"}])
    df.write.mode("overwrite").format("csv").save(d)

    spark.read.schema(df.schema).option(
        "nullValue", "Alice").format('csv').load(d).show()
    # +---+----+
    # |age|name|
    # +---+----+
    # |100|NULL|
    # +---+----+
  • Last updated on