Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Aborts this microbatch due to task failures.
This method is invoked on the driver when one or more tasks failed. The commit messages are collected from the write() method call from each task and passed to this method. The implementation should use the commit messages to abort the microbatch in the streaming sink.
Syntax
abort(messages: List[Optional[WriterCommitMessage]], batchId: int)
Parameters
| Parameter | Type | Description |
|---|---|---|
messages |
list of WriterCommitMessage | A list of commit messages. If a write task fails, the commit message will be None. |
batchId |
int | An integer that uniquely identifies a batch of data being written. The integer increases by 1 with each microbatch processed. |