Merk
Tilgang til denne siden krever autorisasjon. Du kan prøve å logge på eller endre kataloger.
Tilgang til denne siden krever autorisasjon. Du kan prøve å endre kataloger.
Manages all active StreamingQuery instances associated with a SparkSession. Use spark.streams to access this.
Syntax
# Access through SparkSession
spark.streams
Properties
| Property | Description |
|---|---|
active |
Returns a list of all active streaming queries associated with this SparkSession. |
Methods
| Method | Description |
|---|---|
get(id) |
Returns an active query by its unique ID. |
awaitAnyTermination(timeout) |
Waits until any active query terminates, or until the timeout expires. |
resetTerminated() |
Forgets past terminated queries so that awaitAnyTermination() can be used again to wait for new terminations. |
addListener(listener) |
Registers a StreamingQueryListener to receive lifecycle event callbacks. |
removeListener(listener) |
Deregisters a StreamingQueryListener. |
Examples
sdf = spark.readStream.format("rate").load()
sq = sdf.writeStream.format('memory').queryName('this_query').start()
sqm = spark.streams
[q.name for q in sqm.active]
# ['this_query']
sqm.awaitAnyTermination(5)
# True
sq.stop()
sqm.resetTerminated()