option (DataStreamWriter)

Adds an output option for the underlying data source.

Syntax

option(key, value)

Parameters

Returns

DataStreamWriter

Examples

df = spark.readStream.format("rate").load()
df.writeStream.option("x", 1)
# <...streaming.readwriter.DataStreamWriter object ...>

Print 3 rows per batch to the console:

import time
q = spark.readStream.format(
    "rate").option("rowsPerSecond", 10).load().writeStream.format(
        "console").option("numRows", 3).start()
time.sleep(3)
q.stop()