Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Loads a CSV file stream and returns the result as a DataFrame. If inferSchema is enabled, the function goes through the input once to determine the schema. To avoid this pass, disable inferSchema or specify the schema explicitly using schema.
Syntax
csv(path, schema=None, **options)
Parameters
| Parameter | Type | Description |
|---|---|---|
path |
str | Path for the CSV input. |
schema |
StructType or str, optional | Schema as a StructType or DDL-formatted string (for example, col0 INT, col1 DOUBLE). |
Returns
DataFrame
Examples
Load a stream from a temporary CSV file:
import tempfile
import time
with tempfile.TemporaryDirectory(prefix="csv") as d:
spark.createDataFrame([(1, "2"),]).write.mode("overwrite").format("csv").save(d)
q = spark.readStream.schema(
"col0 INT, col1 STRING"
).format("csv").load(d).writeStream.format("console").start()
time.sleep(3)
q.stop()