Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Assigns a name to the streaming source for checkpoint evolution. This allows streaming queries to evolve by enabling sources to be reordered or added without breaking checkpoint compatibility. When source evolution is enabled, all sources in a query must be named.
Syntax
name(source_name)
Parameters
| Parameter | Type | Description |
|---|---|---|
source_name |
str | The name for this streaming source. Must contain only ASCII letters (a-z, A-Z), digits (0-9), and underscores (_). |
Returns
DataStreamReader
Notes
Requires streaming source evolution to be enabled via the spark.sql.streaming.enableSourceEvolution configuration.
Examples
Name two sources and union them:
df1 = spark.readStream.format("rate").name("source1").load()
df2 = spark.readStream.format("rate").name("source2").load()
query = df1.union(df2).writeStream.format("console").start()
Valid and invalid source names:
# Valid names
spark.readStream.format("rate").name("mySource").load()
spark.readStream.format("rate").name("my_source_123").load()
# Invalid name — raises AnalysisException
spark.readStream.format("rate").name("my-source").load()