Share via

outputMode (DataStreamWriter)

Specifies how data of a streaming DataFrame is written to a streaming sink.

Syntax

outputMode(outputMode)

Parameters

Parameter Type Description
outputMode str Output mode. Options are append (only new rows), complete (all rows on every update), and update (only updated rows on every update; equivalent to append if the query contains no aggregations).

Returns

DataStreamWriter

Examples

df = spark.readStream.format("rate").load()
df.writeStream.outputMode('append')
# <...streaming.readwriter.DataStreamWriter object ...>

Use complete mode to print aggregated counts:

import time
df = spark.readStream.format("rate").option("rowsPerSecond", 10).load()
df = df.groupby().count()
q = df.writeStream.outputMode("complete").format("console").start()
time.sleep(3)
q.stop()
  • Last updated on