pyspark.sql.SparkSession.streams

property SparkSession.streams

Returns a StreamingQueryManager that allows managing all the StreamingQuery instances active on this context.

New in version 2.0.0.

Changed in version 3.5.0: Supports Spark Connect.

Returns

StreamingQueryManager

Notes

This API is evolving.

Examples

>>> spark.streams
<pyspark...StreamingQueryManager object ...>

Get the list of active streaming queries

>>> sq = spark.readStream.format(
...     "rate").load().writeStream.format('memory').queryName('this_query').start()
>>> sqm = spark.streams
>>> [q.name for q in sqm.active]
['this_query']
>>> sq.stop()