option (DataStreamWriter)

Adds an output option for the underlying data source.

Syntax

option(key, value)

Parameters

Parameter Type Description
key str The option key.
value str, int, float, or bool The option value.

Returns

DataStreamWriter

Examples

df = spark.readStream.format("rate").load()
df.writeStream.option("x", 1)
# <...streaming.readwriter.DataStreamWriter object ...>

Print 3 rows per batch to the console:

import time
q = spark.readStream.format(
    "rate").option("rowsPerSecond", 10).load().writeStream.format(
        "console").option("numRows", 3).start()
time.sleep(3)
q.stop()
  • Last updated on