Share via

format (DataStreamReader)

Specifies the input data source format.

Syntax

format(source)

Parameters

Parameter Type Description
source str Name of the data source, for example 'json' or 'parquet'.

Returns

DataStreamReader

Examples

spark.readStream.format("text")
# <...streaming.readwriter.DataStreamReader object ...>

Write a text file and read it back as a stream:

import tempfile
import time
with tempfile.TemporaryDirectory(prefix="format") as d:
    spark.createDataFrame(
        [("hello",), ("this",)]).write.mode("overwrite").format("text").save(d)
    q = spark.readStream.format("text").load(d).writeStream.format("console").start()
    time.sleep(3)
    q.stop()
  • Last updated on