text (DataStreamReader)

Loads a text file stream and returns a DataFrame whose schema starts with a string column named value, followed by any partitioned columns. Text files must be encoded as UTF-8. Each line in the text file is a new row in the resulting DataFrame by default.

Syntax

text(path, **options)

Parameters

Returns

DataFrame

Examples

Load a stream from a temporary text file:

import tempfile
import time
with tempfile.TemporaryDirectory(prefix="text") as d:
    spark.createDataFrame(
        [("hello",), ("this",)]).write.mode("overwrite").format("text").save(d)
    q = spark.readStream.text(d).writeStream.format("console").start()
    time.sleep(3)
    q.stop()